{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# German Credit Risk Classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using pre-labeled data from the following Kaggle link:\n",
    "\n",
    "https://www.kaggle.com/uciml/german-credit\n",
    "\n",
    "This is a well known dataset for a classification of consumers into good and bad credit risks. As the dataset comes pre-labelled, a supervised approach is taken and variety of methods are used here to demonstrate the capabilities of different supervised classification algorithms.\n",
    "\n",
    "If the dataset came unlabeled, an unsupervised approach here would be warranted. Clustering algorithms such as K-means or DBScan would be viable options, but likely with similar consumer credit risk data with incomplete data and lack of history for certain consumers, a semi-supervised approach may also be a viable option."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# MISCELLANEOUS IMPORTS\n",
    "import pandas as pd\n",
    "import patsy\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import itertools\n",
    "\n",
    "# MODELLING AND METRICS IMPORTS\n",
    "from sklearn.metrics import classification_report, confusion_matrix, accuracy_score\n",
    "from sklearn.pipeline import Pipeline\n",
    "from sklearn.preprocessing import OneHotEncoder, StandardScaler\n",
    "from sklearn.decomposition import PCA\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.svm import SVC\n",
    "from sklearn.ensemble import RandomForestClassifier, ExtraTreesClassifier\n",
    "from sklearn.naive_bayes import BernoulliNB\n",
    "from sklearn.model_selection import train_test_split, GridSearchCV, cross_val_score\n",
    "from sklearn.linear_model import LogisticRegression, SGDClassifier, PassiveAggressiveClassifier, Perceptron, RidgeClassifier\n",
    "from xgboost import XGBClassifier"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# SET SEABORN STYLE\n",
    "sns.set(context='paper', style='white', palette='bone', font='sansserif', font_scale=1.5, color_codes=False, rc=None)\n",
    "\n",
    "# SET FONT DICTIONARY\n",
    "font = {'family': 'sansserif',\n",
    "        'color':  'black',\n",
    "        'weight': 'normal',\n",
    "        'variant': 'small-caps',\n",
    "        'size': 14\n",
    "        }"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Loading, Exploration and Cleaning"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset contains information on demographic data of the consumers with credit, their financial status indicators and information pertaining to their credit as well. A total of 9 features are given as potential predictors for their credit and we will be using all (after conversion) to predict whether a particular consumer is classified as a good or bad creditor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Job</th>\n",
       "      <th>Housing</th>\n",
       "      <th>Saving accounts</th>\n",
       "      <th>Checking account</th>\n",
       "      <th>Credit amount</th>\n",
       "      <th>Duration</th>\n",
       "      <th>Purpose</th>\n",
       "      <th>Class</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>67</td>\n",
       "      <td>male</td>\n",
       "      <td>2</td>\n",
       "      <td>own</td>\n",
       "      <td>NaN</td>\n",
       "      <td>little</td>\n",
       "      <td>1169</td>\n",
       "      <td>6</td>\n",
       "      <td>radio/TV</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>22</td>\n",
       "      <td>female</td>\n",
       "      <td>2</td>\n",
       "      <td>own</td>\n",
       "      <td>little</td>\n",
       "      <td>moderate</td>\n",
       "      <td>5951</td>\n",
       "      <td>48</td>\n",
       "      <td>radio/TV</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>49</td>\n",
       "      <td>male</td>\n",
       "      <td>1</td>\n",
       "      <td>own</td>\n",
       "      <td>little</td>\n",
       "      <td>NaN</td>\n",
       "      <td>2096</td>\n",
       "      <td>12</td>\n",
       "      <td>education</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>45</td>\n",
       "      <td>male</td>\n",
       "      <td>2</td>\n",
       "      <td>free</td>\n",
       "      <td>little</td>\n",
       "      <td>little</td>\n",
       "      <td>7882</td>\n",
       "      <td>42</td>\n",
       "      <td>furniture/equipment</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>53</td>\n",
       "      <td>male</td>\n",
       "      <td>2</td>\n",
       "      <td>free</td>\n",
       "      <td>little</td>\n",
       "      <td>little</td>\n",
       "      <td>4870</td>\n",
       "      <td>24</td>\n",
       "      <td>car</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Age     Sex  Job Housing Saving accounts Checking account  \\\n",
       "0           0   67    male    2     own             NaN           little   \n",
       "1           1   22  female    2     own          little         moderate   \n",
       "2           2   49    male    1     own          little              NaN   \n",
       "3           3   45    male    2    free          little           little   \n",
       "4           4   53    male    2    free          little           little   \n",
       "\n",
       "   Credit amount  Duration              Purpose  Class  \n",
       "0           1169         6             radio/TV      1  \n",
       "1           5951        48             radio/TV      2  \n",
       "2           2096        12            education      1  \n",
       "3           7882        42  furniture/equipment      1  \n",
       "4           4870        24                  car      2  "
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# HEAD OF DATA FRAME\n",
    "data_raw = pd.read_csv('german_credit_risk.csv')\n",
    "data_raw.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Unnamed: 0            0\n",
       "Age                   0\n",
       "Sex                   0\n",
       "Job                   0\n",
       "Housing               0\n",
       "Saving accounts     183\n",
       "Checking account    394\n",
       "Credit amount         0\n",
       "Duration              0\n",
       "Purpose               0\n",
       "Class                 0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# CHECK FOR NULL VALUES\n",
    "data_raw.isnull().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The data needs to be cleaned and missing values need to be imputed. Features containing string values such as 'male' and 'female' or 'free', 'rent' and 'own' are approriately encoded as binary variables. A one hot encoder is used below to build a matrix that can be plugged into classification algorithms easily.\n",
    "\n",
    "Patsy is an alternative to using a one hot encoder to build a feature dataframe with column names specified, if insight into the weights of particular features are more relevant than the prediction itself."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# DATA CLEANING\n",
    "data = data_raw.ix[:,1:10]\n",
    "label = data_raw.ix[:,10]\n",
    "datan = data\n",
    "datan = datan.replace(['male','female'],[0,1])\n",
    "datan = datan.replace(['free','rent','own'],[0,1,2])\n",
    "datan = datan.replace(['little','moderate','quite rich','rich'],[1,2,3,4])\n",
    "datan = datan.replace(['radio/TV','repairs','domestic appliances','vacation/others','furniture/equipment','car','education','business'],[1,2,3,4,5,6,7,8])\n",
    "datan = datan.fillna(0)\n",
    "enc = OneHotEncoder(categorical_features = [1,2,3,4,5,8],sparse = False)\n",
    "enc.fit(datan)\n",
    "dataprocessed = enc.transform(datan)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>Sex</th>\n",
       "      <th>Job</th>\n",
       "      <th>Housing</th>\n",
       "      <th>Saving accounts</th>\n",
       "      <th>Checking account</th>\n",
       "      <th>Credit amount</th>\n",
       "      <th>Duration</th>\n",
       "      <th>Purpose</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.00000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "      <td>1000.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>35.546000</td>\n",
       "      <td>0.310000</td>\n",
       "      <td>1.904000</td>\n",
       "      <td>1.605000</td>\n",
       "      <td>1.190000</td>\n",
       "      <td>1.06400</td>\n",
       "      <td>3271.258000</td>\n",
       "      <td>20.903000</td>\n",
       "      <td>4.524000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>11.375469</td>\n",
       "      <td>0.462725</td>\n",
       "      <td>0.653614</td>\n",
       "      <td>0.674856</td>\n",
       "      <td>0.966869</td>\n",
       "      <td>1.10776</td>\n",
       "      <td>2822.736876</td>\n",
       "      <td>12.058814</td>\n",
       "      <td>2.442005</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>19.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>250.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>27.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1365.500000</td>\n",
       "      <td>12.000000</td>\n",
       "      <td>1.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>33.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>2319.500000</td>\n",
       "      <td>18.000000</td>\n",
       "      <td>5.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>42.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2.00000</td>\n",
       "      <td>3972.250000</td>\n",
       "      <td>24.000000</td>\n",
       "      <td>6.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>75.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>4.00000</td>\n",
       "      <td>18424.000000</td>\n",
       "      <td>72.000000</td>\n",
       "      <td>8.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               Age          Sex          Job      Housing  Saving accounts  \\\n",
       "count  1000.000000  1000.000000  1000.000000  1000.000000      1000.000000   \n",
       "mean     35.546000     0.310000     1.904000     1.605000         1.190000   \n",
       "std      11.375469     0.462725     0.653614     0.674856         0.966869   \n",
       "min      19.000000     0.000000     0.000000     0.000000         0.000000   \n",
       "25%      27.000000     0.000000     2.000000     1.000000         1.000000   \n",
       "50%      33.000000     0.000000     2.000000     2.000000         1.000000   \n",
       "75%      42.000000     1.000000     2.000000     2.000000         1.000000   \n",
       "max      75.000000     1.000000     3.000000     2.000000         4.000000   \n",
       "\n",
       "       Checking account  Credit amount     Duration      Purpose  \n",
       "count        1000.00000    1000.000000  1000.000000  1000.000000  \n",
       "mean            1.06400    3271.258000    20.903000     4.524000  \n",
       "std             1.10776    2822.736876    12.058814     2.442005  \n",
       "min             0.00000     250.000000     4.000000     1.000000  \n",
       "25%             0.00000    1365.500000    12.000000     1.000000  \n",
       "50%             1.00000    2319.500000    18.000000     5.000000  \n",
       "75%             2.00000    3972.250000    24.000000     6.000000  \n",
       "max             4.00000   18424.000000    72.000000     8.000000  "
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# OVERVIEW OF THE DATA\n",
    "datan.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "While we will be using a cross validated grid search and the dataset is relatively small, it is worthwhile to split the data regardless and use a hold-out test split to test the accuracy of the grid searched predictor on 'real world' data, a test size of 20% will be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TRAIN AND TEST SPLIT WITH TEST SIZE OF 20%\n",
    "data_train,data_test,label_train,label_test = train_test_split(dataprocessed,label,test_size = 0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Modelling with default parameters on multiple classifier algorithms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **K Neighbors** - Uses a preset number of neighbors to classify i.e. if n = 3 the three nearest neighbors to a point are of a single class it will be classified as that particular class.\n",
    "- **Logistic Regression** - Bread and butter of linear classification algorithms. Efficient and robust to noise.\n",
    "- **Decision Trees** - Uses nodes (and child nodes) and questions regarding each feature to build an optimal decision tree to classify.\n",
    "- **Support Vector Machines** - Builds a maximum margin between two boundaries built using support vectors, the maximum margin is only affected by the support vectors and new data not falling on boundary lines do not affect the margin. Also good for high dimensional data as it can use a kernel trick to build a hyperplane for class separation.\n",
    "- **Random Forests** - Ensemble learning method bagging decision tree classifers to classify on the modal class. One of the better working classification algorithms but also prone to overfitting on training data.\n",
    "- **Extra Trees** - Extra trees are similar to random forests except splits are selected at random rather than using a criterion.\n",
    "- **Stochastic Gradient Descent** - Linear classifier using stepwise optimization of loss function given a subset of the training data.\n",
    "- **Perceptron** - A very basic neural network resembling a stochastic gradient descent classifier using heaviside step function.\n",
    "- **Passive Aggressive** - Linear classifier that adjusts the model according to whether a classification is correct or not, in passive, if it is a correct classification it will keep the model, in aggressive, if it is an incorrect classification, the model is updated to adjust to this misclassified example. Also has a regularization parameter.\n",
    "- **Ridge Classifier** - Another linear classifier that uses ridge regularization on a feature set, useful when you want to use all features.\n",
    "- **Bernoulli Naive Bayes** - Naive Bayes is a simple probabilistic classifier based on Bayes theorem which assumes (naively) independence between features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To start we will be throwing a multitude of classifiers on the data to see what performs best with the default parameters setting for the classifiers and also use a principal component analysis on our feature set to reduce the dimensionality of the data, the grid search will find the optimal number of components for each particular algorithm. \n",
    "\n",
    "Note: Standardization is necessary to ensure our features are all on the same scale."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# SET PARAMETER SPACE FOR GRID SEARCH ON PCA COMPONENTS\n",
    "parameters = {\n",
    "'pca__n_components': [2, 4, 8, 10, 20, 25, 29]\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "# LIST OF BINARY CLASSIFIERS\n",
    "models = [KNeighborsClassifier(),\n",
    "          LogisticRegression(),\n",
    "          DecisionTreeClassifier(),\n",
    "          SVC(),\n",
    "          RandomForestClassifier(),\n",
    "          ExtraTreesClassifier(),\n",
    "          SGDClassifier(),\n",
    "          Perceptron(),\n",
    "          PassiveAggressiveClassifier(),\n",
    "          RidgeClassifier(),\n",
    "          BernoulliNB(),\n",
    "          ]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# USED TO PLOT A CONFUSION MATRIX TO CHECK HOW THE CLASSIFICATION MODEL WORKS ON THE TEST SET\n",
    "def plot_confusion_matrix(cm, classes, title='Confusion matrix', cmap=plt.cm.bone):\n",
    "    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n",
    "    plt.title(title, fontdict=font)\n",
    "    tick_marks = np.arange(len(classes))\n",
    "    plt.xticks(tick_marks, classes)\n",
    "    plt.yticks(tick_marks, classes)\n",
    "    thresh = cm.max() / 2.\n",
    "    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n",
    "        plt.text(j, i, cm[i, j],\n",
    "                 horizontalalignment=\"center\",\n",
    "                 color=\"black\" if cm[i, j] > thresh else \"white\",\n",
    "                fontdict=font)\n",
    "    plt.ylabel('True label', fontdict=font, fontsize=10)\n",
    "    plt.xlabel('Predicted label', fontdict=font, fontsize=10)\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    1.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n",
      "           metric_params=None, n_jobs=1, n_neighbors=5, p=2,\n",
      "           weights='uniform')\n",
      "Grid Search Best Score: 0.685\n",
      "Grid Search Best Params: {'pca__n_components': 20}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.77      0.89      0.82       140\n",
      "          2       0.58      0.37      0.45        60\n",
      "\n",
      "avg / total       0.71      0.73      0.71       200\n",
      "\n",
      "Accuracy on Test Data: 0.730\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHzxJREFUeJzt3XlYVPXiP/D3MAPKJlcFlCVJEXFBcCvlhpmk4pIKlmuW\n3mt2vYJmablQVyu3srSyX6So4EWumYbkkkahoqkJCmKIGwSKoIgpyiLLMJ/fHz7O1wkRTD5zZHy/\nnofncc7nzDnvGeTN55wzM6iEEAJERBKYKR2AiEwXC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWzGNk\nwYIFaNGiBVQqFSIjIx96e9nZ2VCpVDh69OjDh2sAnnvuOYSEhCgdo0FR8XUwysrPz8fixYuxY8cO\nXLx4Efb29vD29sa0adMwePDgettPWloaOnfujJiYGPj6+sLOzg6WlpYPtc2qqioUFBTA3t4eGo2m\nnpIaT3Z2Nlq3bo2kpCT06NGj1vWvXbsGc3Nz2NraGiGdaWh4/ytMSHZ2Np555hnY2tpiyZIl8PHx\ngU6nQ3x8PKZMmYILFy7U274yMjIAAIGBgVCpVPWyTbVajZYtW9bLth5lFRUVsLCwQLNmzZSO0vAI\nUsygQYOEk5OTKCoqqjZ27do1/b/Pnz8vAgMDhY2NjbCxsRFBQUEiJydHPz5//nzRqVMnsXHjRtGm\nTRthY2Mjhg8fLgoKCvTjAAy+hBBiwoQJYsiQIQb7vbOtO06cOCH8/f2Fra2tsLGxEd7e3mLPnj1C\nCCGysrIEAJGUlKRfPyEhQTz99NOiUaNGwtHRUcyYMUOUl5frx/v06SP+/e9/i7lz54rmzZsLBwcH\nMXPmTFFVVVXj8xQRESGsra3FDz/8IDw9PYWlpaUYOnSoKCwsFJs3bxZt27YVTZo0EePHjxelpaX6\n++3atUv4+fmJv/3tb6Jp06ZiwIABIj09XT/+5+ekT58+Bs/L0qVLhYuLi3BwcNBnDw4OFkIIcfr0\naWFlZSXWr19vsD9zc3Nx+PDhGh/L44bnYBRy7do17N69GyEhIbCxsak23rRpUwCAEAKBgYHIz8/H\nnj17sHfvXuTl5SEwMBDirqPb7OxsbNq0CVu3bkVcXBxSUlIQGhoKAJg1axbCw8MBAJcuXcKlS5fq\nnHPcuHFwcnJCYmIiUlJSsGDBAjRu3Pie6+bm5mLQoEHo2rUrUlJSsHbtWmzcuBFz5841WC86Ohoa\njQaHDh3Cl19+ic8++wybNm26b47y8nJ8+umniI6ORnx8PI4ePYqXXnoJ69evx3fffYfY2Fjs2LED\nX331lf4+JSUlmDFjBhITE7Fv3z7Y2dlh6NChqKioAAAkJiYCAHbv3o1Lly4hJiZGf9+EhAScOHEC\nu3fvRnx8fLU8np6eWLFiBUJCQpCZmYmCggJMnDgRoaGh6NWrV92e3MeB0g33uDpy5IgAIGJiYu67\nXlxcnDAzMxNZWVn6ZZmZmUKlUomffvpJCHF71tGoUSNRWFioX2fhwoXC3d1df3vz5s3iz9/uusxg\nbG1tRWRk5D2z/XkGM2/ePOHu7m4wG4mIiBAWFhaipKRECHF7FtCrVy+D7fTr109MmjSpxucgIiJC\nABCnT5/WL5s5c6YwMzPTz9Jqejx3Ky4uFmZmZuLAgQP3zH/3duzt7UVZWZnB8rtnMHcEBgaKnj17\nisGDB4u///3vQqvV1rj/xxFnMAoRdTy3furUKTg7O+PJJ5/UL2vTpg2cnZ2Rnp6uX+bm5gY7Ozv9\nbWdnZ1y5cuWhc7711lt47bXX4O/vj0WLFuH06dP3zerr6wszs//7b+Xn54eKigr9OSAA8Pb2Nrhf\nXbI2atQInp6e+tstWrRAy5YtYW9vb7Ds7u1kZmZi3LhxcHd3R5MmTdCiRQvodLo6ndvy8vJCo0aN\nal1vzZo1yMjIwP79+7Fhwwao1epa7/M4YcEoxMPDAyqVCqdOnbrvekKIGk/K3r3c3Ny82phOp7vv\nts3MzKoVXWVlpcHtBQsWID09HYGBgTh06BC8vb2xbt06o2f981UqlUpV63aGDh2KgoICrFq1CkeO\nHEFKSgo0Go3+EOl+rK2ta10HuH117saNGygrK0Nubm6d7vM4YcEopFmzZggICMCXX36J4uLiauOF\nhYUAgI4dOyI3NxfZ2dn6sd9//x15eXno2LHjQ2VwcHCodj7m+PHj1dbz8PDA9OnTsXPnTkyaNAlr\n1qy55/Y6duyIw4cPG/yQ//LLL7CwsIC7u/tDZX1Qf/zxB06dOoV58+ahX79+6NChA4qKiqDVavXr\nWFhYALh9uf2vKCwsxKuvvopZs2YhODgYr7zyCm7evFkv+U0FC0ZBX331FYQQ6NGjBzZv3owzZ87g\n9OnTCAsL0x9G9OvXDz4+Pnj55Zdx7NgxHD16FC+//DK6desGf3//h9q/v78/UlJSsG7dOmRkZODj\njz/GwYMH9eO3bt1CcHAw9u3bh+zsbBw5cgS//PJLjcU2depU5OXlYerUqTh16hR27tyJOXPmICQk\nBFZWVg+V9UE1bdoU9vb2CA8PR0ZGBhISEjBlyhSDmZCjoyMsLS3x448/Ij8/Hzdu3HigfUyZMgX2\n9vb44IMP8NFHH8HW1hbBwcH1/VAaNBaMglq3bo3k5GT0798fs2fPhre3N/z9/bFt2zasWrUKwO1p\nf2xsLBwcHPDcc8+hb9++aNmyJWJjYx/69SwBAQGYP38+QkND0b17d2RnZ2Pq1Kn6cbVajevXr2PC\nhAnw9PREUFAQfH19sXz58ntuz8XFBbt27UJKSgq6dOmCf/7znxg7diwWL178UDn/CjMzM2zatAkn\nTpyAl5cXgoOD8eGHHxqcV9FoNPjiiy+wZs0aODs7Y/jw4XXeflRUFLZt24bo6GiYm5ujUaNG+N//\n/octW7Zg48aNMh5Sg8RX8hKRNJzBEJE0LBgikoYFQ0TSsGCISJpH4t3UZWVlSEtLg4ODA18JSdTA\n3PnYDi8vr2rvU3skCiYtLQ0vv/yy0jGI6CFER0dX+1ydR6JgHBwcANwO+Dh8vsjjqHXr1kpHIEk0\nGg1atWql/zk2GFMgTzV3DotatmwJV1dXhdOQDHe/RJ9M071Ob/AkLxFJw4IhImlYMEQkDQuGiKRh\nwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYM\nEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCR\nNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnD\ngiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGDqyf79+zFs2DC4uLhApVIhMjJSP1ZZWYnZs2fD\n29sb1tbWcHJywrhx43DhwoV7bksIgYEDB0KlUmHLli1GegT0oHr37o3vv/8eFy9ehBACEyZMqLaO\nh4cHvvvuO1y/fh0lJSU4duwY2rdvr0BaZbBg6klxcTG8vLzw+eefw9LS0mCstLQUycnJCA0NRXJy\nMr7//nvk5ORg4MCB0Gq11bb16aefQq1WGys6/UU2NjZIS0vDG2+8gdLS0mrjTz75JA4ePIisrCz4\n+/vDy8sL7777LoqLixVIqxDxCMjJyRHt2rUTOTk5SkepF9bW1iIiIuK+65w8eVIAECdOnDBYnpSU\nJFxdXUV+fr4AIDZv3iwxqfEAMOmvoqIiMWHCBINl0dHRYsOGDYpnk/2l0Whq/PnlDEYhN2/eBAA0\nbdpUv6yoqAhjx47FqlWr4OjoqFQ0qgcqlQpDhw5Feno6du3ahStXriAxMRGjRo1SOppRsWAUUFFR\ngZkzZ2Lo0KFwdXXVL58yZQoGDhyIwYMHK5iO6oOjoyNsbW0xb948xMXFoX///ti4cSOio6MxZMgQ\npeMZjUbpAI8brVaL8ePHo7CwENu2bdMvj4qKQmpqKo4ePapgOqovZma3f3d///33WLFiBQAgNTUV\nPXr0QHBwMHbu3KlkPKOROoPZsGEDRowYAS8vL8yZM0fmrhoErVaLsWPH4sSJE4iPj0fz5s31Y/Hx\n8UhPT4eNjQ00Gg00mtvdP3r0aPj5+SkVmf6iq1evorKyEunp6QbLT506hVatWimUyvikzmAcHR0x\ndepUHDhwAOXl5TJ39cirrKzEmDFjkJaWhn379qFly5YG44sWLcKsWbMMlnXu3BmffPIJhg8fbsyo\nVA8qKyuRlJQET09Pg+Xt2rXD+fPnFUplfFILZsCAAQCA3377Dfn5+TJ3pbji4mJkZGQAAHQ6HS5c\nuIDjx4+jWbNmcHZ2xsiRI5GUlITt27dDpVLh8uXLAAA7OztYWlrCxcUFLi4u1bb7xBNPoE2bNkZ9\nLFQ31tbWaNu2LYDbh0StWrWCj48Prl27hpycHHz88cf49ttvceDAAezZswd9+/bFmDFjEBgYqHBy\nIzLGJcrly5eL2bNn1zhuCpep9+7de89LeBMmTBBZWVk1XuK73+Vs8DL1I/3Vp0+fez7WiIgIg+//\nmTNnRGlpqUhNTRVjxoxRPHd9f93vMjVP8taT5557Drd/ju7tfmP1eR8ynoSEBKhUqvuus379eqxf\nv95IiR49vExNRNKwYIhIGqmHSFqtFlVVVdDpdKiqqkJ5eTnUarX+EiwRmTapM5iwsDB4e3tj9erV\n2LZtG7y9vREWFiZzl0T0CJE6lZg2bRqmTZsmcxdE9AjjORgikoYFQ0TSsGCISBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVD\nRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQk\njaamgZs3b973jk2aNKn3MERkWmosmE6dOkGlUkEIoV9257ZKpcKFCxeMEpCIGq4aCyYnJ8eYOYjI\nBNXpHMw333yDxYsXAwAuXryIY8eOSQ1FRKah1oIJCQnB3r17ERUVBQCwsrLClClTpAcjooavxkOk\nOw4dOoTk5GR07doVANCsWTNUVFRID0ZEDV+tMxhzc3PodDqoVCoAwB9//AEzM17dJqLa1doUwcHB\nePHFF1FQUID58+fDz88Ps2fPNkY2Imrgaj1EevXVV9G9e3f8/PPPAIDNmzfDy8tLejAiavhqLRgA\nqKqqgrm5OVQqFXQ6nexMRGQiaj1EWrRoEcaOHYu8vDxcvHgR48aNw5IlS4yRjYgauFpnMBs2bMCx\nY8dgZWUFAAgNDUX37t0xd+5c6eGIqGGrdQbj5uYGrVarv63VatGmTRupoYjINNQ4g3nzzTehUqlg\nZWWFTp06ISAgACqVCnFxcfDz8zNmRiJqoGosmDtXijp16oQhQ4bol/fq1Ut+KiIyCTUWzKRJk4yZ\ng4hMUK0neTMzMxEaGor09HSUlZXpl589e1ZqMCJq+Go9yTtx4kT84x//gBACu3btwqhRozBmzBhj\nZCOiBq7WgiktLUVAQAAAwN3dHQsXLsTevXulByOihq/WQ6RGjRpBCAF3d3d8/fXXcHFxwZUrV4yR\njYgauFoLZsWKFSguLsYXX3yB0NBQ3LhxA+vWrTNGNiJq4GotmJ49ewIAbG1t9R86RURUFzUWTFBQ\nkP4zYO4lJiZGSiAiMh01FkxISIgxcwAABg0aA5WqTm/wpgbGxqap0hFIErW65mtFNf40P//881LC\nENHjg599SUTSsGCISJo6F0x5ebnMHERkgmotmMTERHTu3BkeHh4AgNTUVEybNk16MCJq+GotmOnT\np2PHjh1o3rw5AMDHx4dvFSCiOqm1YHQ6Hdzc3AyWqdVqaYGIyHTU+qKTJ554AomJiVCpVKiqqsLK\nlSvRrl07Y2Qjogau1hlMWFgYli9fjgsXLqBFixb49ddfERYWZoxsRNTA1TqDcXR0xDfffGOMLERk\nYmotmMmTJ9/zPUmrV6+WEoiITEetBdOvXz/9v8vKyrB161Y88cQTUkMRkWmotWBGjx5tcPuVV15B\n//79pQUiItPxwG8VyMrKwvnz52VkISITU+sMpmnTpvpzMDqdDs2aNcPSpUulByOihu++BSOEQGpq\nKlxcXAAAZmZm9/0QKiKiu933EEmlUiEoKAhqtRpqtZrlQkQPpNZzME8//TSSk5ONkYWITEyNh0ha\nrRYajQa//PILwsPD4e7uDmtrawghoFKpWDpEVKsaC+bOzCU2NtaYeYjIhNRYMEIIALf/miMR0V9R\nY8EUFBRg+fLlNd7xrbfekhKIiExHjQVTVVWF4uJi/UyGiOhB1VgwTk5O+M9//mPMLERkYmq8TM2Z\nCxE9rBoLJj4+3pg5iMgE1VgwzZo1M2YOIjJB/MNrRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoW\nDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQ\nkTQsGCKShgVDRNKwYIhIGhYMEUnDgpFk9OhAbNmyFocO7cShQzsRFfX/0Lt3L/24paUl5syZjp9+\n2ozExB+xbdt/MX78Swompgcxc+YM7Nv3M3JzzyMr6yy+/fZ/6NChg35co9Hggw/m4/DhA7h8OQfn\nzqVj7drVcHV1UTC18bFgJMnPL8Bnn63C6NGvY+zYfyExMRmffbYQHh5tAABvvz0Vzz7bC/PmLUJg\n4ASEh2/AjBmv44UX+iucnOrCz88Pa9asQ79+AzFkyHBotVps3x6Dpk3/BgCwsrKEj48Pli1bjt69\n+2LMmPFwdXXB1q1boFarFU5vPBqlA5iqffsOGtxeuXItRo0aDh+fTjh37nd06eKFHTvikJR0HACQ\nl3cZQUGD0blzR+zY8ZMSkekBBAUZzjYnT/43cnOz0atXT+za9SNu3izC8OEjDNaZPv0tHD16GJ6e\n7ZCefsqYcRXDGYwRmJmZYeBAf1hZWSI1NQ0AkJz8G/r0+TtatHAAAPj4dIKnZ1scPJioZFT6i2xs\nbKBWq1FYWFjjOk2a2AIACgtvGCuW4jiDkcjDozWior6ChYUFSktvYcaM93DuXBYAYOnSL/Dee2/h\np582o7JSq1+2f/9hJSPTX/Txx0uQmnoCR44k3XPc3Nwcixd/iB9+2IW8vDwjp1MOC0airKwcjBz5\nGmxtbdCv37NYuHAuJk2agYyMLIwbNwJdu3ph2rS5yMvLR/fuPpg589/Iy7vMWUwDs2TJQvj69sSA\nAYOh0+mqjavVaqxZswp2dnYYPXqcAgmVI61gKioqsGDBAhw+fBiFhYVwc3PDm2++iT59+sja5SNH\nq9UiJycXAJCefgZeXu0xfvxLWLLkc7zxxmTMnLkACQm3Zyznzv2O9u3bYsKE0SyYBmTJkkV46aUg\nDBkyHNnZ56uNq9VqRESsQadOHTBo0DBcu3ZdgZTKkVYwWq0WTk5OiIqKgrOzMxISEjBjxgxs374d\nrq6usnb7SDMzU8HCwgIajQbm5ubVfttVVVXBzEylUDp6UB99tAQvvRSEwYOH4ezZc9XGNRoNIiPX\nomPH9hg0aBiuXLmiQEplSSsYKysrTJs2TX+7b9++cHV1xcmTJx+Lgnnjjddx4MBhXL5cAGtrSwwa\n1A89enRBSMgclJSUIinpOGbMeB2lpbdw6dJldO/eBUOHBmDFiq+Vjk518OmnH2PMmNEYO3Y8rl8v\nhKOjIwCgpKQEJSUlUKvViIqKQLduXTFq1DgIIfTr3Lx5E2VlZUrGNxqVEEIYY0dXr15F3759ERsb\nC3d3d4Oxixcv4vnnn4eFhQNUKtM4LfThh3Pw1FNdYG/fDMXFJTh79ndERn6DQ4dunwRs3rwZ3nhj\nMnx9e8DOrgkuXcpHTMxOrF+/SeHkcmRlnVA6Qr0qKrp2z+WLF3+EJUs+QqtWT+DkydR7rjNlSjCi\nozfKjGdUarUZWrRojvj4+GqTB6MUTGVlJSZPnoxWrVrhgw8+qDZuigVDhkytYOj/3K9gpL8ORqfT\n4Z133oG5uTnee+892bsjokeI1OmCEAKhoaG4evUqwsPDYW5uLnN3RPSIkVow8+fPR2ZmJiIiItC4\ncWOZuyKiR5C0gsnNzcWmTZtgYWEBPz8//fL3338fw4YNk7VbInqESCsYFxcXnDlzRtbmiagB4Jsd\niUgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aI\npGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEga\nFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHB\nEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwR\nScOCISJpWDBEJI1G6QAAUFVVBQAQokrhJCSLWs3fZabKzOz29/bOz/HdHomCKSgoAABUVl5TOAnJ\n0qJFc6UjkGQFBQVwc3MzWKYSQgiF8uiVlZUhLS0NDg4OUKvVSschogdQVVWFgoICeHl5oXHjxgZj\nj0TBEJFp4oExEUnDgiEiaVgwRCQNC4aIpGHBGNmGDRswYsQIeHl5Yc6cOUrHoXpUUVGBefPmoW/f\nvujatSsCAwORkJCgdCxFPRKvg3mcODo6YurUqThw4ADKy8uVjkP1SKvVwsnJCVFRUXB2dkZCQgJm\nzJiB7du3w9XVVel4imDBGNmAAQMAAL/99hvy8/MVTkP1ycrKCtOmTdPf7tu3L1xdXXHy5MnHtmB4\niEQkydWrV5GdnY22bdsqHUUxLBgiCSorKzFr1iwEBQXB3d1d6TiKYcEQ1TOdTod33nkH5ubmeO+9\n95SOoyiegyGqR0IIhIaG4urVqwgPD4e5ubnSkRTFgjEyrVaLqqoq6HQ6VFVVoby8HGq1GhoNvxWm\nYP78+cjMzERERES1N/49jvhmRyNbuXIlvvzyS4NlISEhBlcfqGHKzc2Fv78/LCwsDH5hvP/++xg2\nbJiCyZTDgiEiaXiSl4ikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgjFRarUaXbp0gZeXF0aOHInS\n0tK/vK19+/bhhRdeAABs27YNS5curXHdwsJCfPXVVw+8jwULFuCTTz6p8/K7TZw4EVu2bKnzvrKz\ns+Hl5fXAGenBsWBMlKWlJY4fP460tDRYWFjg66+/NhgXQkCn0z3wdocNG3bfD8r6qwVDpokF8xjo\n3bs3MjIykJ2djQ4dOmDq1Kno1q0bcnJyEBcXB19fX3Tr1g0jR45EcXExAGD37t1o3749/Pz8EBMT\no99WZGQkQkJCAAD5+fkICgqCj48PfHx8cOjQIcyZMweZmZno0qUL3n77bQDAsmXL8NRTT8Hb2xvz\n58/Xb2vRokXw9PREv379cObMmVofR3h4OJ566in4+PjgxRdfNJiV/fzzz+jduzfatWuHHTt2ALj9\n93refvtt/b5XrVr18E8mPRAWjInTarXYtWsXOnfuDAA4c+YMXn31VaSkpMDa2hoLFy7Ezz//jOTk\nZPTo0QPLly9HWVkZJk+ejO3bt+PAgQO4fPnyPbc9ffp09OnTB6mpqUhOTkanTp2wdOlSuLu74/jx\n41i2bBni4uJw7tw5JCYm4vjx4zh27Bj279+PY8eO4ZtvvkFKSgpiYmKQlJRU62MZMWIEkpKSkJqa\nig4dOmDt2rX6sezsbCQkJGDnzp2YMmUKysrKsHbtWtjZ2SEpKQlJSUkIDw9HVlZW/TyxVCd8h52J\nunXrFrp06QLg9gxm0qRJyMvLg5ubG3r16gUA+PXXX5Geno5nnnkGwO3PlPX19cXp06fRunVreHh4\nAADGjx+P1atXV9vHnj178N///hfA7XM+dnZ2uH79usE6cXFxiIuLQ9euXQEAxcXFOHfuHIqKihAU\nFAQrKysAqNN7ddLS0vDuu++isLAQxcXFCAgI0I+NGjUKZmZm8PDwQJs2bXD69GnExcXhxIkT+vMz\nN27cwLlz59CuXbu6P5H0UFgwJurOOZg/s7a21v9bCIH+/ftj48aNBuscP34cKpWqXnIIITB37lz8\n61//Mlj+2WefPfA+Jk6ciNjYWPj4+CAyMhL79u3Tj/15WyqVCkIIrFy50qCIgNuzHTIOHiI9xnr1\n6oWDBw8iIyMDAFBaWoqzZ8+iffv2yMrKQmZmJgBUK6A7nn/+eYSFhQG4fb7j5s2bsLW1RVFRkX6d\ngIAArFu3Tn9uJzc3F1euXMGzzz6LrVu34tatWygqKsL27dtrzVtUVAQnJydUVlYiOjraYGzz5s3Q\n6XTIzMzE77//Dk9PTwQEBCAsLAyVlZUAgLNnz6KkpOQBnyV6GJzBPMYcHBwQGRmJsWPH6v/CwcKF\nC9GuXTusXr0aQ4YMgb29Pfz8/JCWllbt/p9//jlef/11rF27Fmq1GmFhYfD19cUzzzwDLy8vDBo0\nCMuWLcOpU6fg6+sLALCxscGGDRvQrVs3jB49Gl26dIGbmxt69+5da94PP/wQPXv2hJubGzp37mxQ\nZJ6enujTpw/y8/Px9ddfo3HjxnjttdeQnZ2Nbt26QQgBBwcHxMbG1tOzR3XBj2sgIml4iERE0rBg\niEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKS5v8DvDQV71dd8xkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x117534450>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    0.4s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
      "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
      "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
      "          verbose=0, warm_start=False)\n",
      "Grid Search Best Score: 0.73125\n",
      "Grid Search Best Params: {'pca__n_components': 20}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.76      0.86      0.81       140\n",
      "          2       0.54      0.37      0.44        60\n",
      "\n",
      "avg / total       0.69      0.71      0.70       200\n",
      "\n",
      "Accuracy on Test Data: 0.715\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHvJJREFUeJzt3XtcVGXCB/DfMAMoFwkEFURZRcQUQdRSCiPwwlpp4CVX\nLXXX2jWEVstrbIuVmmVppm+kpOCqma+t4i1dEhXzkqCAhIoIMYKghBkKIpdhnvcP13klRDR55sj4\n+34+fD7Oec6c85tBfjznnJlBJYQQICKSwEzpAERkulgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGAe\nIfPmzUPbtm2hUqkQFxf3wNvTarVQqVQ4fvz4g4drBp599lmEh4crHaNZUfF1MMoqLi7GwoULsXPn\nTly4cAGOjo7w9vZGREQEnnvuuSbbT2ZmJnr27IktW7bAz88PdnZ2aNmy5QNts7a2FiUlJXB0dIRG\no2mipMaj1WrRqVMnpKSkoG/fvo2uf+XKFZibm8PW1tYI6UxD8/tfYUK0Wi2efvpp2Nra4oMPPoCP\njw/0ej0SExMxZcoU5OfnN9m+cnJyAAAhISFQqVRNsk21Wo127do1ybYeZtXV1bCwsICDg4PSUZof\nQYoZOnSocHZ2FmVlZfXGrly5Yvj3+fPnRUhIiLCxsRE2NjYiNDRUFBQUGMajoqJEjx49xMaNG0Xn\nzp2FjY2NePHFF0VJSYlhHECdLyGEmDhxonj++efr7PfWtm7JyMgQQUFBwtbWVtjY2Ahvb2+xb98+\nIYQQeXl5AoBISUkxrJ+UlCSefPJJYWlpKdq0aSOmTZsmqqqqDOMBAQHi9ddfF3PnzhWtW7cWTk5O\n4q233hK1tbUNPk+xsbHC2tpafPvtt8LT01O0bNlSDBs2TJSWlorNmzeLLl26iFatWomXX35ZVFRU\nGO63e/du4e/vLx577DFhb28vhgwZIk6fPm0Y/+1zEhAQUOd5WbRokWjfvr1wcnIyZJ86daoQQois\nrCxhZWUl1q5dW2d/5ubm4ujRow0+lkcNz8Eo5MqVK9izZw/Cw8NhY2NTb9ze3h4AIIRASEgIiouL\nsW/fPuzfvx9FRUUICQmBuO3oVqvVYtOmTdi6dSsSEhKQlpaGyMhIAMCMGTMQExMDALh48SIuXrx4\nzznHjRsHZ2dnJCcnIy0tDfPmzUOLFi3uuG5hYSGGDh0KX19fpKWlYfXq1di4cSPmzp1bZ70NGzZA\no9HgyJEjWLFiBT799FNs2rTprjmqqqrwySefYMOGDUhMTMTx48cxatQorF27Fv/+978RHx+PnTt3\n4vPPPzfc5/r165g2bRqSk5Nx4MAB2NnZYdiwYaiurgYAJCcnAwD27NmDixcvYsuWLYb7JiUlISMj\nA3v27EFiYmK9PJ6enli6dCnCw8ORm5uLkpISTJo0CZGRkejfv/+9PbmPAqUb7lF17NgxAUBs2bLl\nruslJCQIMzMzkZeXZ1iWm5srVCqV+O6774QQN2cdlpaWorS01LDO/Pnzhbu7u+H25s2bxW+/3fcy\ng7G1tRVxcXF3zPbbGczbb78t3N3d68xGYmNjhYWFhbh+/boQ4uYsoH///nW2M2jQIDF58uQGn4PY\n2FgBQGRlZRmWvfXWW8LMzMwwS2vo8dyuvLxcmJmZie+///6O+W/fjqOjo6isrKyz/PYZzC0hISGi\nX79+4rnnnhNPPfWU0Ol0De7/UcQZjELEPZ5bP3PmDFxcXPCHP/zBsKxz585wcXHB6dOnDcvc3Nxg\nZ2dnuO3i4oKff/75gXO++eabePXVVxEUFIQFCxYgKyvrrln9/PxgZvb//638/f1RXV1tOAcEAN7e\n3nXudy9ZLS0t4enpabjdtm1btGvXDo6OjnWW3b6d3NxcjBs3Du7u7mjVqhXatm0LvV5/T+e2vLy8\nYGlp2eh6X375JXJycnDw4EGsX78earW60fs8SlgwCvHw8IBKpcKZM2fuup4QosGTsrcvNzc3rzem\n1+vvum0zM7N6RVdTU1Pn9rx583D69GmEhITgyJEj8Pb2xpo1a4ye9bdXqVQqVaPbGTZsGEpKSrBy\n5UocO3YMaWlp0Gg0hkOku7G2tm50HeDm1bmrV6+isrIShYWF93SfRwkLRiEODg4IDg7GihUrUF5e\nXm+8tLQUANC9e3cUFhZCq9Uaxn766ScUFRWhe/fuD5TBycmp3vmY9PT0eut5eHjgjTfewK5duzB5\n8mR8+eWXd9xe9+7dcfTo0To/5IcOHYKFhQXc3d0fKOv9+uWXX3DmzBm8/fbbGDRoEB5//HGUlZVB\np9MZ1rGwsABw83L771FaWooJEyZgxowZmDp1Kl555RVcu3atSfKbChaMgj7//HMIIdC3b19s3rwZ\nZ8+eRVZWFqKjow2HEYMGDYKPjw/Gjx+PEydO4Pjx4xg/fjx69+6NoKCgB9p/UFAQ0tLSsGbNGuTk\n5OCjjz7C4cOHDeM3btzA1KlTceDAAWi1Whw7dgyHDh1qsNjCwsJQVFSEsLAwnDlzBrt27cKcOXMQ\nHh4OKyurB8p6v+zt7eHo6IiYmBjk5OQgKSkJU6ZMqTMTatOmDVq2bIn//Oc/KC4uxtWrV+9rH1Om\nTIGjoyPee+89fPjhh7C1tcXUqVOb+qE0aywYBXXq1AmpqakYPHgwZs+eDW9vbwQFBWH79u1YuXIl\ngJvT/vj4eDg5OeHZZ59FYGAg2rVrh/j4+Ad+PUtwcDCioqIQGRmJPn36QKvVIiwszDCuVqvx66+/\nYuLEifD09ERoaCj8/PywZMmSO26vffv22L17N9LS0tCrVy/85S9/wdixY7Fw4cIHyvl7mJmZYdOm\nTcjIyICXlxemTp2K999/v855FY1Gg88++wxffvklXFxc8OKLL97z9tetW4ft27djw4YNMDc3h6Wl\nJb766it888032Lhxo4yH1CzxlbxEJA1nMEQkDQuGiKRhwRCRNCwYIpLmoXg3dWVlJTIzM+Hk5MRX\nQhI1M7c+tsPLy6ve+9QeioLJzMzE+PHjlY5BRA9gw4YN9T5X56EoGCcnJwA3Az4Kny/yKOrUqZPS\nEUgSjUaDjh07Gn6O64wpkKeeW4dF7dq1g6urq8JpSIbbX6JPpulOpzd4kpeIpGHBEJE0LBgikoYF\nQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBE\nJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TS\nsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0L\nhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwTeTgwYMYPnw42rdvD5VKhbi4OMNYTU0N\nZs+eDW9vb1hbW8PZ2Rnjxo1Dfn5+nW2sWrUKgYGBeOyxx6BSqaDVao37IOi+DBgwANu2bcOFCxcg\nhMDEiRPrjLdp0waxsbEoLCzE9evXsXv3bnTp0kWhtMpgwTSR8vJyeHl5YdmyZWjZsmWdsYqKCqSm\npiIyMhKpqanYtm0bCgoK8Mc//hE6na7OekOGDMG8efOMnJ5+DxsbG2RmZuLvf/87Kioq6o3Hx8fD\nw8MDISEh8PX1xfnz57F3715YWVkpkFYh4iFQUFAgunbtKgoKCpSO0iSsra1FbGzsXdc5deqUACAy\nMjLqjaWkpAgAIi8vT05ABQAw6a+ysjIxceJEw20PDw8hhBDe3t6GZSqVShQXF4vJkycrnrcpvzQa\nTYM/v5zBKOTatWsAAHt7e4WTkAyWlpYAgMrKSsMyIQSqqqrg7++vVCyjY8EooLq6Gm+99RaGDRsG\nV1dXpeOQBFlZWdBqtVi4cCHs7e1hbm6OWbNmoUOHDnB2dlY6ntGwYIxMp9Ph5ZdfRmlpKWJjY5WO\nQ5LodDqMHDkS7u7uuHLlCioqKhAYGIhvv/0WtbW1SsczGqkFs379eowYMQJeXl6YM2eOzF01Czqd\nDmPHjkVGRgYSExPRunVrpSORRKmpqfD19YWdnR2cnZ0xdOhQtG7dGnl5eUpHMxqpBdOmTRuEhYVh\n5MiRMnfTLNTU1GDMmDHIyMjA/v370a5dO6UjkZFcu3YNly9fRpcuXdC3b19s27ZN6UhGo5G58SFD\nhgAAfvzxRxQXF8vcleLKy8uRk5MDANDr9cjPz0d6ejocHBzg4uKC0aNHIyUlBTt27IBKpcKlS5cA\nAHZ2dobL2pcuXcKlS5eQnZ0NADh9+jRKS0vRsWNHODg4KPPAqEHW1taG17WYmZmhY8eO8PHxwZUr\nV1BQUIBRo0bh8uXLOH/+PHr27Illy5YhPj4e3333ncLJjcgYlyiXLFkiZs+e3eC4KVym3r9//x0v\n4U2cOFHk5eU1eInv9svZUVFRja7TXDX0+JvzV0BAwB0fa2xsrAAgIiIiRH5+vqiqqhJarVa89957\nwtzcXPHcTf11t8vUqv9+86VaunQpiouLsWjRojuOX7hwAQMHDkRiYiKvqpgolUqldASSRKPRoHPn\nznf8+eVVJCKShgVDRNJIPcmr0+lQW1sLvV6P2tpaVFVVQa1WQ6ORulsiekhIncFER0fD29sbq1at\nwvbt2+Ht7Y3o6GiZuySih4jUqURERAQiIiJk7oKIHmI8B0NE0rBgiEgaFgwRScOCISJpWDBEJA0L\nhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCI\nSBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0Lhoik\n0TQ0cO3atbvesVWrVk0ehohMS4MF06NHD6hUKgghDMtu3VapVMjPzzdKQCJqvhosmIKCAmPmICIT\ndE/nYL7++mssXLgQAHDhwgWcOHFCaigiMg2NFkx4eDj279+PdevWAQCsrKwwZcoU6cGIqPlr8BDp\nliNHjiA1NRW+vr4AAAcHB1RXV0sPRkTNX6MzGHNzc+j1eqhUKgDAL7/8AjMzXt0mosY12hRTp07F\nyJEjUVJSgqioKPj7+2P27NnGyEZEzVyjh0gTJkxAnz59sHfvXgDA5s2b4eXlJT0YETV/jRYMANTW\n1sLc3BwqlQp6vV52JiIyEY0eIi1YsABjx45FUVERLly4gHHjxuGDDz4wRjYiauYancGsX78eJ06c\ngJWVFQAgMjISffr0wdy5c6WHI6LmrdEZjJubG3Q6neG2TqdD586dpYYiItPQ4Axm+vTpUKlUsLKy\nQo8ePRAcHAyVSoWEhAT4+/sbMyMRNVMNFsytK0U9evTA888/b1jev39/+amIyCQ0WDCTJ082Zg4i\nMkGNnuTNzc1FZGQkTp8+jcrKSsPy7OxsqcGIqPlr9CTvpEmT8Oc//xlCCOzevRsvvfQS/vSnPxkj\nGxE1c40WTEVFBYKDgwEA7u7umD9/Pvbv3y89GBE1f40eIllaWkIIAXd3d3zxxRdo3749fv75Z2Nk\nI6JmrtGCWbp0KcrLy/HZZ58hMjISV69exZo1a4yRjYiauUYLpl+/fgAAW1tbw4dOERHdiwYLJjQ0\n1PAZMHeyZcsWKYGIyHQ0WDDh4eHGzAEAGDx4JFQqtdH3S/JZWLRQOgJJotE0/DPbYMEMHDhQShgi\nenTwsy+JSBoWDBFJc88FU1VVJTMHEZmgRgsmOTkZPXv2hIeHBwDg5MmTiIiIkB6MiJq/RgvmjTfe\nwM6dO9G6dWsAgI+PD98qQET3pNGC0ev1cHNzq7NMrealZCJqXKOv5O3QoQOSk5OhUqlQW1uL5cuX\no2vXrsbIRkTNXKMzmOjoaCxZsgT5+flo27YtfvjhB0RHRxsjGxE1c43OYNq0aYOvv/7aGFmIyMQ0\nWjCvvfbaHd+TtGrVKimBiMh0NFowgwYNMvy7srISW7duRYcOHaSGIiLT0GjBjBkzps7tV155BYMH\nD5YWiIhMx32/VSAvLw/nz5+XkYWITEyjMxh7e3vDORi9Xg8HBwcsWrRIejAiav7uWjBCCJw8eRLt\n27cHAJiZmd31Q6iIiG5310MklUqF0NBQqNVqqNVqlgsR3ZdGz8E8+eSTSE1NNUYWIjIxDR4i6XQ6\naDQaHDp0CDExMXB3d4e1tTWEEFCpVCwdImpUgwVza+YSHx9vzDxEZEIaLBghBICbf82RiOj3aLBg\nSkpKsGTJkgbv+Oabb0oJRESmo8GCqa2tRXl5uWEmQ0R0vxosGGdnZ/zzn/80ZhYiMjENXqbmzIWI\nHlSDBZOYmGjMHERkghosGAcHB2PmICITxD+8RkTSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCR\nNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnD\ngiEiaVgwRCQNC4aIpGHBEJE0LBhJxo4dga1b/4Xk5O+QnPwdvvpqFZ555inDuJVVS0RGTse+ffFI\nTd2PXbs2YsKEMQompvsxc+YMHD58CCUlxbhwIR9btnyD7t27G8Y1Gg0WLJiP48eTceXKZWi1P2Ht\n2jh06NBBwdTGx4KRpLi4BEuWfI5RoyZh9Oi/4NixE1i+fBG6dnUHAMya9QaeeeYpzJnzHl54YSxW\nrlyLN998HcOG/VHh5HQvAgKewcqVKxEQEIjg4KHQ6Wqxe/cu2NvbAwCsrKzg69sLixZ9hP79/TBq\n1EtwdXXFjh3boFarFU5vPCrxEPwR6gsXLmDgwIEwM3sMKpXpPvlHj+7B0qXR+N//3YZt29bju+/2\nY8WK1YbxtWv/B9nZuViwYImCKeXIzU1TOoJU1tbWKCkpxujRL2HXrm/vuE63bt1w8mQaevfui1On\nThk5oTwajRquru2RmJgIV1fXOmOcwRiBmZkZhg4dBCurlkhL+xEAkJp6Es8+64927doAAHr18kK3\nbh44dOgHJaPS72Rrawu1Wo1ffy1tcJ1WrVoBAEpLG17H1GiUDmDKPDw6Y+PGVbCwsEBFxQ1ERMzF\nuXM/AQAWLlyKqKhZ2LcvHjU1uv8uW4KkpCNKRqbf6ZNPPkZ6ejp++OHOvyDMzc3x4YeLsHPnThQW\nFho5nXJYMBJptfkYMWIibG1tMWTIs/jgg39g4sRw5OT8hPHjR8PXtyfCwmaiqOgS+vbthZkzw1FY\neBGHDh1TOjrdh48++hBPPfUUgoKCoNfr642r1WrExcXiscfsMHLkKAUSKkdawVRXV2PevHk4evQo\nSktL4ebmhunTpyMgIEDWLh86NTU65Off/G116lQWvLwex8SJYzB//ieYPn0Kpk//Bw4cOAwAyM7O\nRbduHvjzn8exYJqRxYs/wujRoxAc/Efk5WnrjavVaqxb9y94efXA4MHBuHLlivFDKkhaweh0Ojg7\nO2PdunVwcXFBUlISpk2bhh07dtQ7EfSoUKnMYGFhAY1GA3Nzc9TW1tYZ1+v1UKl4Wqy5+OSTjzF6\n9CgMGRKMs2ez641rNBqsX78OPXp0x+DBwSguLlYgpbKkFYyVlRUiIiIMtwMDA+Hq6opTp049EgUz\nffrrOHjwCC5eLIa1tRVeeGEInnzSF6+/PgPXr1cgOTkVb74ZhoqKGygquoQnnvDF8OFD8ckn/6N0\ndLoHy5Ytxbhx4zB69Ev49ddStG3bFgBQXl6O69evQ61WY+PGr9CnTx+MGDESQgjDOlevXkVlZaWS\n8Y3GaJepL1++jMDAQMTHx8Pd3b3OmClepl6wIBL9+vWBo6MDysquIzs7B2vWfIXDh28e/jg6OmD6\n9Nfx1FNPws6uFYqKLuHf/96O2NiNCieXw9QuU1dV3bjj8vffn4/58xfAza0jsrPP3nGdV199DevW\nrZcZz6judpnaKAVTU1OD1157DR07dsR7771Xb9wUC4bqMrWCof+n6Otg9Ho9Zs2aBXNzc7zzzjuy\nd0dEDxGpl6mFEIiMjMTly5cRExMDc3NzmbsjooeM1IKJiopCbm4uYmNj0aJFC5m7IqKHkLSCKSws\nxKZNm2BhYQF/f3/D8nfffRfDhw+XtVsieohIK5j27dvj7Nk7n0UnokcDX9VFRNKwYIhIGhYMEUnD\ngiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwY\nIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEi\naVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKG\nBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaTRK\nBwCA2tpaAIAQtQonIVk0GrXSEUgStfrm9/bWz/HtHoqCKSkpAQAIUQYhFA5DUri6tlc6AklWUlIC\nNze3OstUQij/I11ZWYnMzEw4OTkZ2pCImofa2lqUlJTAy8sLLVq0qDP2UBQMEZkmnuQlImlYMEQk\nDQuGiKRhwRCRNCwYI1u/fj1GjBgBLy8vzJkzR+k41ISqq6vx9ttvIzAwEL6+vggJCUFSUpLSsRT1\nULwO5lHSpk0bhIWF4fvvv0dVVZXScagJ6XQ6ODs7Y926dXBxcUFSUhKmTZuGHTt2wNXVVel4imDB\nGNmQIUMAAD/++COKi4sVTkNNycrKChEREYbbgYGBcHV1xalTpx7ZguEhEpEkly9fhlarRZcuXZSO\nohgWDJEENTU1mDFjBkJDQ+Hu7q50HMWwYIiamF6vx6xZs2Bubo533nlH6TiK4jkYoiYkhEBkZCQu\nX76MmJgYmJubKx1JUSwYI9PpdKitrYVer0dtbS2qqqqgVquh0fBbYQqioqKQm5uL2NjYem/8exTx\nzY5Gtnz5cqxYsaLOsvDw8DpXH6h5KiwsRFBQECwsLOr8wnj33XcxfPhwBZMphwVDRNLwJC8RScOC\nISJpWDBEJA0LhoikYcEQkTQsGCKShgVjotRqNXr16gUvLy+MHj0aFRUVv3tbBw4cwAsvvAAA2L59\nOxYtWtTguqWlpfj888/vex/z5s3Dxx9/fM/Lbzdp0iR8880397wvrVYLLy+v+85I948FY6JatmyJ\n9PR0ZGZmwsLCAl988UWdcSEE9Hr9fW93+PDhd/2grN9bMGSaWDCPgAEDBiAnJwdarRaPP/44wsLC\n0Lt3bxQUFCAhIQF+fn7o3bs3Ro8ejfLycgDAnj170K1bN/j7+2PLli2GbcXFxSE8PBwAUFxcjNDQ\nUPj4+MDHxwdHjhzBnDlzkJubi169emHmzJkAgMWLF+OJJ56At7c3oqKiDNtasGABPD09MWjQIJw9\ne7bRxxETE4MnnngCPj4+GDlyZJ1Z2d69ezFgwAB07doVO3fuBHDz7/XMnDnTsO+VK1c++JNJ94UF\nY+J0Oh12796Nnj17AgDOnj2LCRMmIC0tDdbW1pg/fz727t2L1NRU9O3bF0uWLEFlZSVee+017Nix\nA99//z0uXbp0x22/8cYbCAgIwMmTJ5GamooePXpg0aJFcHd3R3p6OhYvXoyEhAScO3cOycnJSE9P\nx4kTJ3Dw4EGcOHECX3/9NdLS0rBlyxakpKQ0+lhGjBiBlJQUnDx5Eo8//jhWr15tGNNqtUhKSsKu\nXbswZcoUVFZWYvXq1bCzs0NKSgpSUlIQExODvLy8pnli6Z7wHXYm6saNG+jVqxeAmzOYyZMno6io\nCG5ubujfvz8A4IcffsDp06fx9NNPA7j5mbJ+fn7IyspCp06d4OHhAQB4+eWXsWrVqnr72LdvH/71\nr38BuHnOx87ODr/++muddRISEpCQkABfX18AQHl5Oc6dO4eysjKEhobCysoKAO7pvTqZmZn4xz/+\ngdLSUpSXlyM4ONgw9tJLL8HMzAweHh7o3LkzsrKykJCQgIyMDMP5matXr+LcuXPo2rXrvT+R9EBY\nMCbq1jmY37K2tjb8WwiBwYMHY+PGjXXWSU9Ph0qlapIcQgjMnTsXf/vb3+os//TTT+97H5MmTUJ8\nfDx8fHwQFxeHAwcOGMZ+uy2VSgUhBJYvX16niICbsx0yDh4iPcL69++Pw4cPIycnBwBQUVGB7Oxs\ndOvWDXl5ecjNzQWAegV0y8CBAxEdHQ3g5vmOa9euwdbWFmVlZYZ1goODsWbNGsO5ncLCQvz88894\n5plnsHXrVty4cQNlZWXYsWNHo3nLysrg7OyMmpoabNiwoc7Y5s2bodfrkZubi59++gmenp4IDg5G\ndHQ0ampqAADZ2dm4fv36fT5L9CA4g3mEOTk5IS4uDmPHjjX8hYP58+eja9euWLVqFZ5//nk4OjrC\n398fmZmZ9e6/bNky/PWvf8Xq1auhVqsRHR0NPz8/PP300/Dy8sLQoUOxePFinDlzBn5+fgAAGxsb\nrF+/Hr1798aYMWPQq1cvuLm5YcCAAY3mff/999GvXz+4ubmhZ8+edYrM09MTAQEBKC4uxhdffIEW\nLVrg1VdfhVarRe/evSGEgJOTE+Lj45vo2aN7wY9rICJpeIhERNKwYIhIGhYMEUnDgiEiaVgwRCQN\nC4aIpGHBEJE0LBgikub/AG97CWPbxE7lAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11787c810>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    0.7s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model DecisionTreeClassifier(class_weight=None, criterion='gini', max_depth=None,\n",
      "            max_features=None, max_leaf_nodes=None,\n",
      "            min_impurity_split=1e-07, min_samples_leaf=1,\n",
      "            min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
      "            presort=False, random_state=None, splitter='best')\n",
      "Grid Search Best Score: 0.67125\n",
      "Grid Search Best Params: {'pca__n_components': 25}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.77      0.79      0.78       140\n",
      "          2       0.47      0.45      0.46        60\n",
      "\n",
      "avg / total       0.68      0.69      0.68       200\n",
      "\n",
      "Accuracy on Test Data: 0.685\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHw5JREFUeJzt3X1czXfjP/DX6XR/M5NON9i6aIRS7hUZEl3mZsXG3Ayb\n2Vwqcxkb2hbGZnMNm34yoQwXLlvr62ZcTcjtlOQm96UzKroZRSjVef/+8HCutaRM73Pq9Ho+Hh4P\n5/P+nM/n1cHL+/Puc04KIYQAEZEERvoOQESGiwVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC6YBmTt3\nLhwcHKBQKBAdHf3Mx1Or1VAoFDh+/Pizh6sH+vTpg+DgYH3HqFcUvA9Gv3JycvD5559jx44dyMzM\nhJ2dHTw8PBASEoJXXnml1s6TmpqK9u3bIyYmBt7e3mjUqBEsLCye6Zjl5eXIy8uDnZ0djI2Naymp\n7qjVarRo0QJJSUno0qVLtfvfvHkTJiYmsLGx0UE6w1D//lYYELVajZ49e8LGxgZffPEFPD09odFo\nEB8fj8mTJ+Pq1au1dq60tDQAQEBAABQKRa0cU6lUwtHRsVaOVZc9ePAApqamsLW11XeU+keQ3gwc\nOFA4OTmJO3fuVBq7efOm9ve//fabCAgIENbW1sLa2loEBgaKa9euacfDwsKEm5ub2LRpk2jZsqWw\ntrYWr776qsjLy9OOA6jwSwghxo8fLwYNGlThvI+O9cjp06eFr6+vsLGxEdbW1sLDw0Ps3btXCCFE\nRkaGACCSkpK0+yckJIhu3boJMzMzYW9vL6ZNmyZKSkq047179xb/+Mc/xOzZs0WTJk2ESqUSH3zw\ngSgvL6/ydYqKihJWVlbi559/Fq6ursLCwkIMGTJEFBQUiK1bt4qXXnpJPPfcc2Ls2LHi3r172uft\n2rVL+Pj4iOeff140btxYDBgwQJw7d047/ufXpHfv3hVel0WLFolmzZoJlUqlzR4UFCSEEOLChQvC\n0tJSrFu3rsL5TExMxNGjR6v8WhoarsHoyc2bN7F7924EBwfD2tq60njjxo0BAEIIBAQEICcnB3v3\n7sW+ffuQnZ2NgIAAiD9c3arVamzZsgU//fQT4uLikJKSgtDQUADAjBkzEBkZCQC4fv06rl+/XuOc\no0ePhpOTExITE5GSkoK5c+fC3Nz8sftmZWVh4MCB6NixI1JSUrBmzRps2rQJs2fPrrDfxo0bYWxs\njCNHjiA8PBzLli3Dli1bnpijpKQEX3/9NTZu3Ij4+HgcP34cr732GtatW4cff/wRsbGx2LFjB1as\nWKF9zt27dzFt2jQkJiZi//79aNSoEYYMGYIHDx4AABITEwEAu3fvxvXr1xETE6N9bkJCAk6fPo3d\nu3cjPj6+Uh5XV1csXboUwcHBSE9PR15eHiZMmIDQ0FB4eXnV7MVtCPTdcA3VsWPHBAARExPzxP3i\n4uKEkZGRyMjI0G5LT08XCoVC/PLLL0KIh7MOMzMzUVBQoN1nwYIFwsXFRft469at4s9/3DWZwdjY\n2Ijo6OjHZvvzDGbOnDnCxcWlwmwkKipKmJqairt37wohHs4CvLy8KhzHz89PTJw4scrXICoqSgAQ\nFy5c0G774IMPhJGRkXaWVtXX80dFRUXCyMhIHDx48LH5/3gcOzs7UVxcXGH7H2cwjwQEBIju3buL\nV155RfTo0UOUlZVVef6GiDMYPRE1XFs/f/48mjZtir/97W/abS1btkTTpk1x7tw57TZnZ2c0atRI\n+7hp06bIzc195pzTp0/HO++8A19fXyxcuBAXLlx4YlZvb28YGf3vr5WPjw8ePHigXQMCAA8PjwrP\nq0lWMzMzuLq6ah87ODjA0dERdnZ2Fbb98Tjp6ekYPXo0XFxc8Nxzz8HBwQEajaZGa1vu7u4wMzOr\ndr/Vq1cjLS0NBw4cwIYNG6BUKqt9TkPCgtGTVq1aQaFQ4Pz580/cTwhR5aLsH7ebmJhUGtNoNE88\ntpGRUaWiKy0trfB47ty5OHfuHAICAnDkyBF4eHhg7dq1Os/65+9SKRSKao8zZMgQ5OXl4bvvvsOx\nY8eQkpICY2Nj7SXSk1hZWVW7D/Dwu3OFhYUoLi5GVlZWjZ7TkLBg9MTW1hb+/v4IDw9HUVFRpfGC\nggIAQLt27ZCVlQW1Wq0du3LlCrKzs9GuXbtnyqBSqSqtx5w8ebLSfq1atcLUqVOxc+dOTJw4EatX\nr37s8dq1a4ejR49W+Ed+6NAhmJqawsXF5ZmyPq3ff/8d58+fx5w5c+Dn54e2bdvizp07KCsr0+5j\namoK4OG32/+KgoICjBs3DjNmzEBQUBDefPNN3L59u1byGwoWjB6tWLECQgh06dIFW7duxcWLF3Hh\nwgVERERoLyP8/Pzg6emJMWPGIDk5GcePH8eYMWPQqVMn+Pr6PtP5fX19kZKSgrVr1yItLQ1fffUV\nDh8+rB2/f/8+goKCsH//fqjVahw7dgyHDh2qstimTJmC7OxsTJkyBefPn8fOnTsxa9YsBAcHw9LS\n8pmyPq3GjRvDzs4OkZGRSEtLQ0JCAiZPnlxhJmRvbw8LCwv897//RU5ODgoLC5/qHJMnT4adnR3m\nz5+PL7/8EjY2NggKCqrtL6VeY8HoUYsWLXDixAn0798fH330ETw8PODr64tt27bhu+++A/Bw2h8b\nGwuVSoU+ffqgb9++cHR0RGxs7DPfz+Lv74+wsDCEhoaic+fOUKvVmDJlinZcqVTi1q1bGD9+PFxd\nXREYGAhvb28sWbLkscdr1qwZdu3ahZSUFHTo0AFvv/02Ro0ahc8///yZcv4VRkZG2LJlC06fPg13\nd3cEBQXhs88+q7CuYmxsjG+//RarV69G06ZN8eqrr9b4+OvXr8e2bduwceNGmJiYwMzMDP/+97/x\nww8/YNOmTTK+pHqJd/ISkTScwRCRNCwYIpKGBUNE0rBgiEiaOvFu6uLiYqSmpkKlUvFOSKJ65tHH\ndri7u1d6n1qdKJjU1FSMGTNG3zGI6Bls3Lix0ufq1ImCUalUAB4GbAifL9IQubo+213HVHcplUo4\nOdlr/x3/UZ0omEeXRY6OjmjevLme05AMf/V2fKo/Hre8wUVeIpKGBUNE0rBgiEgaFgwRScOCISJp\nWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYF\nQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBE\nJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TS\nsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwdSiAwcOYOjQoWjWrBkUCgWio6MrjMfExMDf3x8qlQoK\nhQL79++vdIySkhKEhITAzs4OVlZWGDp0KDIzM3XzBVCNvffeu0hKOobc3OvIzb2O/fv34u9/96+w\nz8cfz8GVK2m4dSsfcXG70LZtWz2l1R8WTC0qKiqCu7s7vvnmG1hYWFQav3v3Lnr06IElS5ZUeYxp\n06bhxx9/xKZNm3Dw4EHcvn0bgwcPRnl5uczo9JSysrIQGvoJvLx6okePXkhISMDWrVvg7u4OAPjg\ng+l4//2pmD79A/Ts+TJyc/Owc+d2WFtb6zm5bimEEELfITIzM9GvXz/Ex8ejefPm+o5TK6ytrREe\nHo4JEyZUGsvPz4dKpcK+ffvQp08f7fbCwkKoVCpERUVhzJgxAIBr167B2dkZu3btgr+/f6Vj1Rfm\n5lb6jiBddvY1fPppGFavXouMjHSsXLkSX365GABgbm6Oa9fUmD17DlavXqvnpLVLqVSieXOnx/77\n5QymDklOTkZpaSkGDBig3fbCCy+gbdu2OHLkiB6T0ZMYGRnh9ddfg7W1NY4ePYYWLf4GJydH7NkT\nr92nuLgYhw4dhpeXl95y6oOxvgPQ/9y4cQNKpRJ2dnYVtjs4OODGjRt6SkVVcXNzQ0LCXpibm6Oo\nqAgjRryBs2fPwsurOwAgNze3wv65ublo2rSpPqLqDQumHhBCQKFQ6DsG/cmlS5fQrZs3nn++EQID\nA7B69SoMGDBQO/7n1QeFQlFpm6GTeom0YcMGDBs2DO7u7pg1a5bMUxkER0dHlJeXIz8/v8L23Nxc\nODg46CkVVaW0tBRXrlzBiRMp+OSTMJw+fQZTpwYjJycHACr9malUqkqzGkMntWDs7e0xZcoUDB8+\nXOZpDEbnzp1hYmKCX375RbstMzMT58+fR48ePfSYjGrCyMgIZmZmyMhQ4/r1G+jXz1c7ZmZmhp49\ne+DXX3/VY0Ldk3qJ9Gix8syZM9pWN2RFRUVIS0sDAGg0Gly9ehUnT56Era0tXnzxRdy8eRNXr15F\nQUEBACAtLQ3PP/88HB0d4ejoiEaNGmHixImYOXMm7O3t0aRJE0yfPh0eHh7w8/PT55dGf/LZZ/Ox\ne/duZGZmwtraBm+8MQIvv9wLAQEP/zMND/9/+Oijmbh48RIuX07DrFkfoqjoLjZv/o+ek+sW12Bq\n0fHjx9G3b1/t47CwMISFhWH8+PGIjo7Gtm3b8NZbb2nHJ02apN1v7ty5AIClS5fC2NgYI0eOxP37\n99GvXz98//33UCqVOv1a6MkcHR0QFbUGDg4OKCy8jdTUVAwdGog9e/YAAL7+egksLMyxbNlSNG78\nPJKSkjB48FAUFRXpOblu6eQ+mKVLlyInJweLFi167Lgh3gdDFTWE+2AaKt4HQ0R6wYIhImmkrsGU\nlZWhvLwcGo0G5eXlKCkpgVKphLExl36IGgKpM5iIiAh4eHhg1apV2LZtGzw8PBARESHzlERUh0id\nSoSEhCAkJETmKYioDuMaDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOC\nISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgi\nkoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpLGuKqB27dvP/GJzz33XK2HISLD\nUmXBuLm5QaFQQAih3fbosUKhwNWrV3USkIjqryoL5tq1a7rMQUQGqEZrMJs3b8bnn38OAMjMzERy\ncrLUUERkGKotmODgYOzbtw/r168HAFhaWmLy5MnSgxFR/VflJdIjR44cwYkTJ9CxY0cAgK2tLR48\neCA9GBHVf9XOYExMTKDRaKBQKAAAv//+O4yM+N1tIqpetU0RFBSE4cOHIy8vD2FhYfDx8cFHH32k\ni2xEVM9Ve4k0btw4dO7cGXv27AEAbN26Fe7u7tKDEVH9V23BAEB5eTlMTEygUCig0WhkZyIiA1Ht\nJdLChQsxatQoZGdnIzMzE6NHj8YXX3yhi2xEVM9VO4PZsGEDkpOTYWlpCQAIDQ1F586dMXv2bOnh\niKh+q3YG4+zsjLKyMu3jsrIytGzZUmooIjIMVc5g/vnPf0KhUMDS0hJubm7w9/eHQqFAXFwcfHx8\ndJmRiOqpKgvm0XeK3NzcMGjQIO12Ly8v+amIyCBUWTATJ07UZQ4iMkDVLvKmp6cjNDQU586dQ3Fx\nsXb7pUuXpAYjovqv2kXeCRMm4K233oIQArt27cKIESPwxhtv6CIbEdVz1RbMvXv34O/vDwBwcXHB\nggULsG/fPunBiKj+q/YSyczMDEIIuLi4YOXKlWjWrBlyc3N1kY2I6rlqC2bp0qUoKirCt99+i9DQ\nUBQWFmLt2rW6yEZE9Vy1BdO9e3cAgI2NjfZDp4iIaqLKggkMDNR+BszjxMTESAlERIajyoIJDg7W\nZQ4AgIdHF75b20CVlNzTdwSSxNi46guhKkf69esnJQwRNRz87EsikoYFQ0TS1LhgSkpKZOYgIgNU\nbcEkJiaiffv2aNWqFQDg1KlTCAkJkR6MiOq/agtm6tSp2LFjB5o0aQIA8PT05FsFiKhGqi0YjUYD\nZ2fnCtuUSqW0QERkOKq9k/eFF15AYmIiFAoFysvLsXz5crRu3VoX2Yionqt2BhMREYElS5bg6tWr\ncHBwwK+//oqIiAhdZCOieq7aGYy9vT02b96siyxEZGCqLZhJkyY99j1Jq1atkhKIiAxHtQXj5+en\n/X1xcTF++uknvPDCC1JDEZFhqLZgRo4cWeHxm2++if79+0sLRESG46nfKpCRkYHffvtNRhYiMjDV\nzmAaN26sXYPRaDSwtbXFokWLpAcjovrviQUjhMCpU6fQrFkzAICRkdETP4SKiOiPnniJpFAoEBgY\nCKVSCaVSyXIhoqdS7RpMt27dcOLECV1kISIDU+UlUllZGYyNjXHo0CFERkbCxcUFVlZWEEJAoVCw\ndIioWlUWzKOZS2xsrC7zEJEBqbJghBAAHv40RyKiv6LKgsnLy8OSJUuqfOL06dOlBCIiw1FlwZSX\nl6OoqEg7kyEielpVFoyTkxM+/fRTXWYhIgNT5bepOXMhomdVZcHEx8frMgcRGaAqC8bW1laXOYjI\nAPEHrxGRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhI\nGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQtGknfe\neRuHD+/H1avpuHo1HXFxP2PAAD/teGjoR0hMPIysrAyo1Zfwf//3A7p166rHxPQ0Zs2ahcTERBQW\nFiI3Nxfbtm2Dm5tbhX2EEI/9FR4erqfUuseCkSQ7OxthYfPRu7cf+vbtjwMHDmLjxnVwc2sHALh8\nOR0zZsxCjx598Pe/D8Fvv13FDz9shkql0nNyqok+ffpgxYoV6NGjB3x9fVFWVoY9e/agcePG2n0c\nHR0r/Bo8eDAA4D//+Y++YuucQtSBH0KdmZmJfv36IS+vEBqNRt9xpMnIuIh58xYiOvr7SmM2Nta4\ndu0Khg0bib179+khnVyFhXn6jiCVlZUVCgsLERAQgB07djx2n1WrVuHll19GmzZtdJxOLmNjY7Rs\n2RLx8fFo3rx5hTHOYHTAyMgIw4YFwMrKComJSZXGTUxMMH78OBQW3saZM6l6SEjPysbGBkqlErdu\n3XrsuJWVFd544w1ERkbqOJl+Ges7gCFr164t4uJ+hrm5Ge7evYuxYyfg3Lnz2nF///5Ys2YVLC0t\ncONGDgIDX0denmH/T2+ovvnmG6SkpODo0aOPHR89ejTMzMywbt06HSfTL85gJLp8OQ29evnCz28g\n1qyJRkTEcrRt+7/p8cGDh9Grly8GDBiE+Pi9iI6OhIODvR4T01/x9ddfw8fHB8OHD6/yEn/SpEmI\njY1Ffn6+jtPpl7SCefDgAebMmYO+ffuiY8eOCAgIQEJCgqzT1UmlpaXIyMjAyZOnMH/+Qpw5cxZT\nprynHb937x4yMjJw/HgyQkL+idLSMowbN1aPielpLVmyBKNGjYKvry8yMjIeu4+npye6du3a4C6P\nAIkFU1ZWBicnJ6xfvx7Jycl4//33MW3aNGRmZso6ZZ1nZKSAqanZE8aNYGpqqsNE9CyWLVuG0aNH\nw9fXFxcvXqxyv3fffRcZGRnYs2ePDtPVDdLWYCwtLRESEqJ93LdvXzRv3hxnz56ttNJsiMLCPkZc\n3C/IysqGtbU1XnttGHx8emLEiNGwsbHG1KnB2L07Djk5OWjSpAkmTXobTZs6ITZ2m76jUw2Eh4fj\nzTffREBAAG7dugUHBwcAQFFREe7evavdz8LCAmPGjMFXX32lr6h6pbNF3vz8fKjVarz00ku6OqVe\nOTjYY9WqFbC3t8ft27dx9ux5vPbaKOzduw8WFhZo27YNxo4dDVvbxrh58xZSUlLwyiuv4uzZc/qO\nTjUQFBQEANi7d2+F7XPnzsW8efO0j0eOHAkrKytERUXpNF9doZP7YEpLSzFp0iS8+OKLmD9/fqXx\nhnIfTENm6PfBNGR6vQ9Go9Hgww8/hImJCT755BPZpyOiOkTqJZIQAqGhocjPz0dkZCRMTExkno6I\n6hipBRMWFob09HRERUXB3Nxc5qmIqA6SVjBZWVnYsmULTE1N4ePjo90+b948DB06VNZpiagOkVYw\nzZo1e+K9AURk+PhWASKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TS\nsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0L\nhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCI\nSBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0Lhoik\nYcEQkTQsGCKShgVDRNKwYIhIGhYMEUljrO8AAFBeXg4AMDJSgJ1nmIyN68RfNZLg0Z/to3/HFcZ0\nHeZx8vLyAABNmjyn5yQki0rVSN8RSLK8vDw4OztX2KYQQgg95dEqLi5GamoqVCoVlEqlvuMQ0VMo\nLy9HXl4e3N3dYW5uXmGsThQMERkmLngQkTQsGCKShgVDRNKwYIhIGhaMjm3YsAHDhg2Du7s7Zs2a\npe84VIsePHiAOXPmoG/fvujYsSMCAgKQkJCg71h6VSfug2lI7O3tMWXKFBw8eBAlJSX6jkO1qKys\nDE5OTli/fj2aNm2KhIQETJs2Ddu3b0fz5s31HU8vWDA6NmDAAADAmTNnkJOTo+c0VJssLS0REhKi\nfdy3b180b94cZ8+ebbAFw0skIkny8/OhVqvx0ksv6TuK3rBgiCQoLS3FjBkzEBgYCBcXF33H0RsW\nDFEt02g0+PDDD2FiYoJPPvlE33H0imswRLVICIHQ0FDk5+cjMjISJiYm+o6kVywYHSsrK0N5eTk0\nGg3Ky8tRUlICpVLJjzMwEGFhYUhPT0dUVFSlN/41RHyzo44tX74c4eHhFbYFBwdX+O4D1U9ZWVnw\n9fWFqalphf8w5s2bh6FDh+oxmf6wYIhIGi7yEpE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMAZK\nqVSiQ4cOcHd3x+uvv4579+795WPt378fgwcPBgBs27YNixYtqnLfgoICrFix4qnPMXfuXPzrX/+q\n8fY/mjBhAn744Ycan0utVsPd3f2pM9LTY8EYKAsLC5w8eRKpqakwNTXFypUrK4wLIaDRaJ76uEOH\nDn3iB2X91YIhw8SCaQB69eqFtLQ0qNVqtG3bFlOmTEGnTp1w7do1xMXFwdvbG506dcLrr7+OoqIi\nAMDu3bvRpk0b+Pj4ICYmRnus6OhoBAcHAwBycnIQGBgIT09PeHp64siRI5g1axbS09PRoUMHzJw5\nEwCwePFidO3aFR4eHggLC9Mea+HChXB1dYWfnx8uXrxY7dcRGRmJrl27wtPTE8OHD68wK9uzZw96\n9eqF1q1bY8eOHQAe/ryemTNnas/93XffPfuLSU+FBWPgysrKsGvXLrRv3x4AcPHiRYwbNw4pKSmw\nsrLCggULsGfPHpw4cQJdunTBkiVLUFxcjEmTJmH79u04ePAgbty48dhjT506Fb1798apU6dw4sQJ\nuLm5YdGiRXBxccHJkyexePFixMXF4fLly0hMTMTJkyeRnJyMAwcOIDk5GZs3b0ZKSgpiYmKQlJRU\n7dcybNgwJCUl4dSpU2jbti3WrFmjHVOr1UhISMDOnTsxefJkFBcXY82aNWjUqBGSkpKQlJSEyMhI\nZGRk1M4LSzXCd9gZqPv376NDhw4AHs5gJk6ciOzsbDg7O8PLywsA8Ouvv+LcuXPo2bMngIefKevt\n7Y0LFy6gRYsWaNWqFQBg7NixWLVqVaVz7N27F99//z2Ah2s+jRo1wq1btyrsExcXh7i4OHTs2BEA\nUFRUhMuXL+POnTsIDAyEpaUlANTovTqpqan4+OOPUVBQgKKiIvj7+2vHRowYASMjI7Rq1QotW7bE\nhQsXEBcXh9OnT2vXZwoLC3H58mW0bt265i8kPRMWjIF6tAbzZ1ZWVtrfCyHQv39/bNq0qcI+J0+e\nhEKhqJUcQgjMnj0b7733XoXty5Yte+pzTJgwAbGxsfD09ER0dDT279+vHfvzsRQKBYQQWL58eYUi\nAh7Odkg3eInUgHl5eeHw4cNIS0sDANy7dw+XLl1CmzZtkJGRgfT0dACoVECP9OvXDxEREQAernfc\nvn0bNjY2uHPnjnYff39/rF27Vru2k5WVhdzcXLz88sv46aefcP/+fdy5cwfbt2+vNu+dO3fg5OSE\n0tJSbNy4scLY1q1bodFokJ6ejitXrsDV1RX+/v6IiIhAaWkpAODSpUu4e/fuU75K9Cw4g2nAVCoV\noqOjMWrUKO1POFiwYAFat26NVatWYdCgQbCzs4OPjw9SU1MrPf+bb77Bu+++izVr1kCpVCIiIgLe\n3t7o2bMn3N3dMXDgQCxevBjnz5+Ht7c3AMDa2hobNmxAp06dMHLkSHTo0AHOzs7o1atXtXk/++wz\ndO/eHc7Ozmjfvn2FInN1dUXv3r2Rk5ODlStXwtzcHO+88w7UajU6deoEIQRUKhViY2Nr6dWjmuDH\nNRCRNLxEIiJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnz/wGQ9ZvT7ogaDwAAAABJ\nRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11758f2d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    1.4s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n",
      "  decision_function_shape=None, degree=3, gamma='auto', kernel='rbf',\n",
      "  max_iter=-1, probability=False, random_state=None, shrinking=True,\n",
      "  tol=0.001, verbose=False)\n",
      "Grid Search Best Score: 0.73625\n",
      "Grid Search Best Params: {'pca__n_components': 29}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.75      0.91      0.82       140\n",
      "          2       0.59      0.28      0.38        60\n",
      "\n",
      "avg / total       0.70      0.72      0.69       200\n",
      "\n",
      "Accuracy on Test Data: 0.725\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHmxJREFUeJzt3XlcVOXiP/DPMCyyaSKgIMpVRDJHUGyRxOsSRqUoWGbu\nmi2G2LW0TMm00uJeSyt7SYoKpVwXknDpauQCapaggIriAjEpoIgLChLLMM/3D3/Oz0kRTJ45Ap/3\n68VL5jxnzvnMKB+fc87MoBJCCBARSWCmdAAiarxYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgmpB5\n8+ahdevWUKlUiImJue/tabVaqFQqHDx48P7DNQD9+vVDWFiY0jEaFBVfB6OswsJCfPLJJ9i6dSvy\n8vLg6OgIb29vTJ06Fc8991y97SczMxPdunVDfHw8/Pz80KJFC1hbW9/XNqurq1FUVARHR0eYm5vX\nU1LT0Wq16NChA1JTU/Hoo4/Wuv7ly5dhYWEBe3t7E6RrHBrev4pGRKvVonfv3rC3t8enn34KHx8f\n6PV67Ny5E5MnT8aZM2fqbV/Z2dkAgODgYKhUqnrZplqtRps2beplWw+yyspKWFpawsHBQekoDY8g\nxTz77LPCxcVFlJSU3DZ2+fJlw/d//PGHCA4OFnZ2dsLOzk6EhISIs2fPGsbnzp0runbtKtauXSs6\nduwo7OzsxNChQ0VRUZFhHIDRlxBCjB8/XgwaNMhovze3ddORI0fEgAEDhL29vbCzsxPe3t5i165d\nQgghcnNzBQCRmppqWD85OVk8/vjjwsrKSjg7O4tp06aJiooKw3jfvn3FG2+8IWbNmiVatWolnJyc\nxPTp00V1dXWNz1N0dLSwtbUV//vf/4SXl5ewtrYWQUFBori4WMTFxYlOnTqJ5s2bizFjxoiysjLD\n/bZt2yb8/f3FQw89JFq2bCmefvppcfz4ccP4X5+Tvn37Gj0vERERom3btsLJycmQfcqUKUIIIU6c\nOCFsbGzEt99+a7Q/CwsL8euvv9b4WJoanoNRyOXLl7F9+3aEhYXBzs7utvGWLVsCAIQQCA4ORmFh\nIXbt2oXdu3ejoKAAwcHBELcc3Wq1Wqxfvx4//PADEhMTkZ6ejvDwcADAjBkzEBUVBQA4d+4czp07\nV+eco0aNgouLC1JSUpCeno558+ahWbNmd1w3Pz8fzz77LHr06IH09HSsXLkSa9euxaxZs4zWi42N\nhbm5Ofbv34+vv/4aX3zxBdavX3/XHBUVFfj8888RGxuLnTt34uDBg3jhhRfw7bffYuPGjUhISMDW\nrVuxdOlSw32uX7+OadOmISUlBUlJSWjRogWCgoJQWVkJAEhJSQEAbN++HefOnUN8fLzhvsnJyThy\n5Ai2b9+OnTt33pbHy8sLixcvRlhYGHJyclBUVIQJEyYgPDwcvXr1qtuT2xQo3XBN1YEDBwQAER8f\nf9f1EhMThZmZmcjNzTUsy8nJESqVSvz8889CiBuzDisrK1FcXGxYZ/78+cLDw8NwOy4uTvz1r7su\nMxh7e3sRExNzx2x/ncHMnj1beHh4GM1GoqOjhaWlpbh+/boQ4sYsoFevXkbbCQgIEJMmTarxOYiO\njhYAxIkTJwzLpk+fLszMzAyztJoez61KS0uFmZmZ2Lt37x3z37odR0dHUV5ebrT81hnMTcHBweKJ\nJ54Qzz33nHjyySeFTqercf9NEWcwChF1PLeelZUFV1dX/OMf/zAs69ixI1xdXXH8+HHDMnd3d7Ro\n0cJw29XVFRcuXLjvnG+//TZeeeUVDBgwAAsWLMCJEyfumtXPzw9mZv//n5W/vz8qKysN54AAwNvb\n2+h+dclqZWUFLy8vw+3WrVujTZs2cHR0NFp263ZycnIwatQoeHh4oHnz5mjdujX0en2dzm1pNBpY\nWVnVut6KFSuQnZ2NPXv2YM2aNVCr1bXepylhwSjE09MTKpUKWVlZd11PCFHjSdlbl1tYWNw2ptfr\n77ptMzOz24quqqrK6Pa8efNw/PhxBAcHY//+/fD29saqVatMnvWvV6lUKlWt2wkKCkJRURGWLVuG\nAwcOID09Hebm5oZDpLuxtbWtdR3gxtW5q1evory8HPn5+XW6T1PCglGIg4MDAgMD8fXXX6O0tPS2\n8eLiYgDAI488gvz8fGi1WsPY77//joKCAjzyyCP3lcHJyem28zEZGRm3refp6Yk333wTP/74IyZN\nmoQVK1bccXuPPPIIfv31V6Mf8n379sHS0hIeHh73lfVeXbp0CVlZWZg9ezYCAgLQpUsXlJSUQKfT\nGdaxtLQEcONy+99RXFyMcePGYcaMGZgyZQrGjh2La9eu1Uv+xoIFo6ClS5dCCIFHH30UcXFxOHny\nJE6cOIHIyEjDYURAQAB8fHwwevRoHDp0CAcPHsTo0aPh6+uLAQMG3Nf+BwwYgPT0dKxatQrZ2dn4\nz3/+g19++cUw/ueff2LKlClISkqCVqvFgQMHsG/fvhqLLTQ0FAUFBQgNDUVWVhZ+/PFHvPfeewgL\nC4ONjc19Zb1XLVu2hKOjI6KiopCdnY3k5GRMnjzZaCbk7OwMa2tr/PTTTygsLMTVq1fvaR+TJ0+G\no6MjPvroI/z73/+Gvb09pkyZUt8PpUFjwSioQ4cOSEtLw8CBAzFz5kx4e3tjwIAB2Lx5M5YtWwbg\nxrQ/ISEBTk5O6NevH/r37482bdogISHhvl/PEhgYiLlz5yI8PBw9e/aEVqtFaGioYVytVuPKlSsY\nP348vLy8EBISAj8/PyxatOiO22vbti22bduG9PR0dO/eHS+//DJGjhyJTz755L5y/h1mZmZYv349\njhw5Ao1GgylTpuDjjz82Oq9ibm6Or776CitWrICrqyuGDh1a5+2vXr0amzdvRmxsLCwsLGBlZYX/\n/ve/+P7777F27VoZD6lB4it5iUgazmCISBoWDBFJw4IhImlYMEQkzQPxbury8nJkZmbCycmJr4Qk\namBufmyHRqO57X1qD0TBZGZmYvTo0UrHIKL7EBsbe9vn6jwQBePk5ATgRsCm8PkiTVGHDh2UjkCS\nmJubo3379oafY6MxBfLc5uZhUZs2beDm5qZwGpLh1pfoU+N0p9MbPMlLRNKwYIhIGhYMEUnDgiEi\naVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKG\nBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgw\nRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE\n0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsmHqyZ88eDBkyBG3btoVKpUJMTIxhrKqqCjNn\nzoS3tzdsbW3h4uKCUaNG4cyZM0bbOH/+PMaOHYs2bdrA1tYWPj4+iI2NNfEjobrq06cPNm3ahLy8\nPAghMH78eMOYubk5IiIicPjwYZSWlqKgoACxsbFo166dgolNjwVTT0pLS6HRaPDll1/C2traaKys\nrAxpaWkIDw9HWloaNm3ahLNnz+KZZ56BTqczrDdu3DhkZWVh06ZNOHr0KMaNG4exY8diz549pn44\nVAd2dnbIzMzEv/71L5SVlRmN2djYwNfXFwsWLICvry+GDh2Kdu3aYfv27VCr1QolVoB4AJw9e1Z0\n7txZnD17Vuko9cLW1lZER0ffdZ1jx44JAOLIkSNG91u1apXReu3btxcLFy6UEdOkADTqr5KSEjF+\n/Pi7rtOlSxchhBAajUbxvPX5ZW5uXuPPL2cwCrl27RoAoGXLloZl/v7+2LBhAy5dugS9Xo9Nmzah\nqKgIAQEBSsWketS8eXMAwJUrVxROYjosGAVUVlZi+vTpCAoKgpubm2H5hg0boFKp4OjoCCsrK4we\nPRpr165F9+7dFUxL9cHCwgKff/45Nm/ejPz8fKXjmIy50gGaGp1OhzFjxqC4uBibN282Gnv//fdx\n8eJF7NixA46OjkhISMC4ceOwZ88e+Pj4KJSY7pdarcaaNWvw0EMPYciQIUrHMSmpBbNmzRrEx8fj\n1KlTGDx4MCIiImTu7oGn0+kwcuRIHD16FElJSWjVqpVhLCcnB0uWLEFGRoahTHx8fLB3714sWbIE\nK1asUCo23Qe1Wo21a9eiW7du6NevHy5fvqx0JJOSWjDOzs4IDQ3F3r17UVFRIXNXD7yqqiq89NJL\nyMzMRFJSEtq0aWM0fvMqxF+vMKjVauj1epPlpPpjbm6OdevWQaPRoF+/figsLFQ6kslJLZinn34a\nAHD06NFG/+SWlpYiOzsbAKDX63HmzBlkZGTAwcEBrq6uGD58OFJTU7FlyxaoVCqcP38eANCiRQtY\nW1vj4YcfRqdOnRAaGorPPvsMrVq1QkJCAn7++Wds2rRJyYdGNbC1tUWnTp0AAGZmZmjfvj18fHxw\n+fJlFBQUIC4uDo899hiCgoIghEDr1q0BAFevXkV5ebmS0U3HFJcoFy1aJGbOnFnjeGO4TL179+47\nXsIbP368yM3NrfES362Xs0+dOiWGDRsmnJ2dhY2NjfD29hYxMTHKPah6VNPjb8hfffv2veNjjY6O\nFu7u7jU+F7Vdzm5oX3e7TM2TvPWkX79+uPFzdGd3G7vJ09MTGzdurM9YJFFycjJUKlWN43cbayp4\nmZqIpGHBEJE0Ug+RdDodqqurodfrUV1djYqKCqjVapib88iMqCmQOoOJjIyEt7c3li9fjs2bN8Pb\n2xuRkZEyd0lEDxCpU4mpU6di6tSpMndBRA8wnoMhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBg\niEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aI\npGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0pjXNHDt\n2rW73rF58+b1HoaIGpcaC6Zr165QqVQQQhiW3bytUqlw5swZkwQkooarxoI5e/asKXMQUSNUp3Mw\n69atwyeffAIAyMvLw6FDh6SGIqLGodaCCQsLw+7du7F69WoAgI2NDSZPniw9GBE1fDUeIt20f/9+\npKWloUePHgAABwcHVFZWSg9GRA1frTMYCwsL6PV6qFQqAMClS5dgZsar20RUu1qbYsqUKXj++edR\nVFSEuXPnwt/fHzNnzjRFNiJq4Go9RBo3bhx69uyJHTt2AADi4uKg0WikByOihq/WggGA6upqWFhY\nQKVSQa/Xy85ERI1ErYdICxYswMiRI1FQUIC8vDyMGjUKn376qSmyEVEDV+sMZs2aNTh06BBsbGwA\nAOHh4ejZsydmzZolPRwRNWy1zmDc3d2h0+kMt3U6HTp27Cg1FBE1DjXOYN566y2oVCrY2Niga9eu\nCAwMhEqlQmJiIvz9/U2ZkYgaqBoL5uaVoq5du2LQoEGG5b169ZKfiogahRoLZtKkSabMQUSNUK0n\neXNychAeHo7jx4+jvLzcsPzUqVNSgxFRw1frSd4JEyZg4sSJEEJg27ZtePHFF/HSSy+ZIhsRNXC1\nFkxZWRkCAwMBAB4eHpg/fz52794tPRgRNXy1HiJZWVlBCAEPDw988803aNu2LS5cuGCKbETUwNVa\nMIsXL0ZpaSm++uorhIeH4+rVq1i1apUpshFRA1drwTzxxBMAAHt7e8OHThER1UWNBRMSEmL4DJg7\niY+PlxKIiBqPGgsmLCzMlDkAAMOHvwG12tLk+yX5rK3tlY5AkqjV6hrHaiyYp556SkoYImo6+NmX\nRCQNC4aIpKlzwVRUVMjMQUSNUK0Fk5KSgm7dusHT0xMAcPjwYUydOlV6MCJq+GotmDfffBNbt25F\nq1atAAA+Pj58qwAR1UmtBaPX6+Hu7m607G6XpYiIbqr1lbzt2rVDSkoKVCoVqqursWTJEnTu3NkU\n2Yiogat1BhMZGYlFixbhzJkzaN26NX777TdERkaaIhsRNXC1zmCcnZ2xbt06U2Qhokam1oJ59dVX\n7/iepOXLl0sJRESNR60FExAQYPi+vLwcP/zwA9q1ayc1FBE1DrUWzIgRI4xujx07FgMHDpQWiIga\nj3t+q0Bubi7++OMPGVmIqJGpdQbTsmVLwzkYvV4PBwcHRERESA9GRA3fXQtGCIHDhw+jbdu2AAAz\nM7O7fggVEdGt7nqIpFKpEBISArVaDbVazXIhontS6zmYxx9/HGlpaabIQkSNTI2HSDqdDubm5ti3\nbx+ioqLg4eEBW1tbCCGgUqlYOkRUqxoL5ubMJSEhwZR5iKgRqbFghBAAbvw2RyKiv6PGgikqKsKi\nRYtqvOPbb78tJRARNR41Fkx1dTVKS0sNMxkiontVY8G4uLjggw8+MGUWImpkarxMzZkLEd2vGgtm\n586dpsxBRI1QjQXj4OBgyhxE1AjxF68RkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aI\npGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEga\nFgwRScOCISJpWDBEJA0LxkTGjh2Gffs24q23XjEse+WVlxAb+xV+/jkW27Z9iy++mAuNxkvBlFRX\nvXs/ibi4dcjOPoGysmsYM2aU0XhZ2bU7fi1e/LlCiZVR4++mpvrTtasngoICkJ2tNVp+5kwBFi2K\nwrlzF2BlZYkXXxyMzz9/Hy+9FIYrV64qE5bqxM7ODsePH0ds7FqsWLHstvEOHToZ3fb19cXGjRuw\ncWO8qSI+EFgwktna2uCDD6YhImIpJk4cbjSWmLjH6PaSJTEICgqAp2cHpKRkmDIm3aOffkrETz8l\nAgCWL4+8bbyw8ILR7cGDn8OpU6exb98vJsn3oOAhkmTvvjsZSUm/Ii0t867rmZubY+jQgSgtvY7T\np3NNlI5MwdbWFi+88Dyio2OUjmJynMFIFBQUADe3Nvj4469qXOfJJ3ti3ry30KyZFS5duoK33vqI\nh0eNzIgRw2FlZYXY2P8qHcXkOIORpF07V7z22ih8+OGX0Ol0Na6XlpaJiRNn4I03ZuPAgQx89NF0\ntGr1kAmTkmwTJ07Ali1bcfHiJaWjmJy0gqmsrMTs2bPRv39/9OjRA8HBwUhOTpa1uweORuOFli1b\n4LvvFiMpaQOSkjagRw8NQkICkZS0ARYWNyaP5eUVyM8/j2PHTiMiYil0umoMHhygcHqqL97e3dCz\npy+io79VOooipB0i6XQ6uLi4YPXq1XB1dUVycjKmTZuGLVu2wM3NTdZuHxh79x7A2LHZRstmzw5D\nXt45fPfdRlRV3XlWY2amgqWlhSkikgm8/PJEaLVa7Nq1W+koipBWMDY2Npg6darhdv/+/eHm5oZj\nx441iYIpLS1DaWmZ0bLy8nJcu1aK3NyzsLGxxujRwfjll4O4dOkKHnqoOYYNewZOTq2wa9d+hVJT\nXdna2sLDoyMAwMzMDO3atYO3dzdcvnwFeXl5AABra2uMGDEcixd/qWRURZnsJO/Fixeh1WrRqVOn\n2lduAqqrq9GhQzsMGjQAzZvb49q1EmRlZSMsbA5ycv5QOh7Vwte3B3766X+G23PmhGPOnHCsXh2L\n119/AwDwwgvDYGtri9Wr1ygVU3EqIYSQvZOqqiq8+uqraN++PT766KPbxvPy8vDUU0/BwaEz1GpL\n2XFIAWlpPysdgSRRq9VwdXXGzp07bzs6kX4VSa/X491334WFhQXmzJkje3dE9ACReogkhEB4eDgu\nXryIqKgoWFjw5CVRUyK1YObOnYucnBxER0ejWbNmMndFRA8gaQWTn5+P9evXw9LSEv7+/oblH374\nIYYMGSJrt0T0AJFWMG3btsXJkydlbZ6IGgC+VYCIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4Ih\nImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVD\nRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSmCsdAACqq6v/359VCichWdRq\ntdIRSBK1+sY85ebP8a0eiIIpKioCAFy9mqtwEpLF1dVZ6QgkWVFREdzd3Y2WqYQQQqE8BuXl5cjM\nzISTkxP/pyNqYKqrq1FUVASNRoNmzZoZjT0QBUNEjRNP8hKRNCwYIpKGBUNE0rBgiEgaFoyJrVmz\nBsOGDYNGo8F7772ndByqR5WVlZg9ezb69++PHj16IDg4GMnJyUrHUtQD8TqYpsTZ2RmhoaHYu3cv\nKioqlI5D9Uin08HFxQWrV6+Gq6srkpOTMW3aNGzZsgVubm5Kx1MEC8bEnn76aQDA0aNHUVhYqHAa\nqk82NjaYOnWq4Xb//v3h5uaGY8eONdmC4SESkSQXL16EVqtFp06dlI6iGBYMkQRVVVWYMWMGQkJC\n4OHhoXQcxbBgiOqZXq/Hu+++CwsLC8yZM0fpOIriORiieiSEQHh4OC5evIioqChYWFgoHUlRLBgT\n0+l0qK6uhl6vR3V1NSoqKqBWq2Fuzr+KxmDu3LnIyclBdHT0bW/8a4r4ZkcTW7JkCb7++mujZWFh\nYUZXH6hhys/Px4ABA2BpaWn0H8aHH36IIUOGKJhMOSwYIpKGJ3mJSBoWDBFJw4IhImlYMEQkDQuG\niKRhwRCRNCyYRkqtVqN79+7QaDQYPnw4ysrK/va2kpKSMHjwYADA5s2bERERUeO6xcXFWLp06T3v\nY968efjss8/qvPxWEyZMwPfff1/nfWm1Wmg0mnvOSPeOBdNIWVtbIyMjA5mZmbC0tMQ333xjNC6E\ngF6vv+ftDhky5K4flPV3C4YaJxZME9CnTx9kZ2dDq9WiS5cuCA0Nha+vL86ePYvExET4+fnB19cX\nw4cPR2lpKQBg+/btePjhh+Hv74/4+HjDtmJiYhAWFgYAKCwsREhICHx8fODj44P9+/fjvffeQ05O\nDrp374533nkHALBw4UI89thj8Pb2xty5cw3bWrBgAby8vBAQEICTJ0/W+jiioqLw2GOPwcfHB88/\n/7zRrGzHjh3o06cPOnfujK1btwK48ft63nnnHcO+ly1bdv9PJt0TFkwjp9PpsG3bNnTr1g0AcPLk\nSYwbNw7p6emwtbXF/PnzsWPHDqSlpeHRRx/FokWLUF5ejldffRVbtmzB3r17cf78+Ttu+80330Tf\nvn1x+PBhpKWloWvXroiIiICHhwcyMjKwcOFCJCYm4vTp00hJSUFGRgYOHTqEPXv24NChQ1i3bh3S\n09MRHx+P1NTUWh/LsGHDkJqaisOHD6NLly5YuXKlYUyr1SI5ORk//vgjJk+ejPLycqxcuRItWrRA\namoqUlNTERUVhdxc/vZQU+I77BqpP//8E927dwdwYwYzadIkFBQUwN3dHb169QIA/Pbbbzh+/Dh6\n9+4N4MZnyvr5+eHEiRPo0KEDPD09AQBjxozB8uXLb9vHrl278N133wG4cc6nRYsWuHLlitE6iYmJ\nSExMRI8ePQAApaWlOH36NEpKShASEgIbGxsAqNN7dTIzM/H++++juLgYpaWlCAwMNIy9+OKLMDMz\ng6enJzp27IgTJ04gMTERR44cMZyfuXr1Kk6fPo3OnTvX/Ymk+8KCaaRunoP5K1tbW8P3QggMHDgQ\na9euNVonIyMDKpWqXnIIITBr1iy8/vrrRsu/+OKLe97HhAkTkJCQAB8fH8TExCApKckw9tdtqVQq\nCCGwZMkSoyICbsx2yDR4iNSE9erVC7/88guys7MBAGVlZTh16hQefvhh5ObmIicnBwBuK6Cbnnrq\nKURGRgK4cb7j2rVrsLe3R0lJiWGdwMBArFq1ynBuJz8/HxcuXMA///lP/PDDD/jzzz9RUlKCLVu2\n1Jq3pKQELi4uqKqqQmxsrNFYXFwc9Ho9cnJy8Pvvv8PLywuBgYGIjIxEVVUVAODUqVO4fv36PT5L\ndD84g2nCnJycEBMTg5EjRxp+w8H8+fPRuXNnLF++HIMGDYKjoyP8/f2RmZl52/2//PJLvPbaa1i5\nciXUajUiIyPh5+eH3r17Q6PR4Nlnn8XChQuRlZUFPz8/AICdnR3WrFkDX19fjBgxAt27d4e7uzv6\n9OlTa96PP/4YTzzxBNzd3dGtWzejIvPy8kLfvn1RWFiIb775Bs2aNcMrr7wCrVYLX19fCCHg5OSE\nhISEenr2qC74cQ1EJA0PkYhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TS/B83Xcjo\nP6u82AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1153c4890>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    2.5s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
      "            max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
      "            min_impurity_split=1e-07, min_samples_leaf=1,\n",
      "            min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
      "            n_estimators=10, n_jobs=1, oob_score=False, random_state=None,\n",
      "            verbose=0, warm_start=False)\n",
      "Grid Search Best Score: 0.72625\n",
      "Grid Search Best Params: {'pca__n_components': 25}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.72      0.87      0.79       140\n",
      "          2       0.40      0.20      0.27        60\n",
      "\n",
      "avg / total       0.62      0.67      0.63       200\n",
      "\n",
      "Accuracy on Test Data: 0.670\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHulJREFUeJzt3XlU1OXiP/D3MOyLJoLK4uUKKS4sipqSeFU0yUyDuu6W\nlFkoYJamGdcLmhplaYsnVFLpiqnHLpFacMkF1DQhARVxCXJSQREXlEWWYZ7fH36dXxMimDwzAu/X\nOZ3jPM9nPp/3jPDu+XxmUSGEECAiksDI0AGIqOViwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCaUWi\noqLQsWNHKBQKxMXFPfT+VCoVFAoFfvnll4cP1wwMHToUYWFhho7RrCj4PhjDKioqwvLly7Fr1y5c\nvHgRdnZ28PLyQnh4OJ555pkmO05OTg48PT2RkJAAX19ftG3bFhYWFg+1z9raWhQXF8POzg7GxsZN\nlFR/VCoVunTpgoyMDPTr16/B7a9fvw4TExPY2NjoIV3L0Px+KloQlUqFQYMGwcbGBu+//z68vb2h\n0WiwZ88ehISE4Pz58012rLy8PABAYGAgFApFk+xTqVSiU6dOTbKvR1l1dTVMTU1ha2tr6CjNjyCD\nGTVqlHBwcBClpaV15q5fv6798++//y4CAwOFtbW1sLa2FkFBQeLChQva+cjISNGrVy+xZcsW4erq\nKqytrcVzzz0niouLtfMAdP4TQohp06aJ0aNH6xz37r7uOn78uPD39xc2NjbC2tpaeHl5ib179woh\nhDh37pwAIDIyMrTbp6WliSeeeEKYmZmJDh06iDlz5oiqqirt/JAhQ8TMmTPFwoULRfv27YW9vb2Y\nO3euqK2trfd52rhxo7CyshI//PCDcHd3FxYWFmLMmDGipKREbN++XTz++OOiTZs2YurUqaKiokJ7\nv6SkJOHn5ycee+wx0a5dOzFy5EiRm5urnf/zczJkyBCd5yU6Olo4OTkJe3t7bfbQ0FAhhBCnT58W\nlpaW4quvvtI5nomJiTh8+HC9j6W14TUYA7l+/TqSk5MRFhYGa2vrOvPt2rUDAAghEBgYiKKiIuzd\nuxf79u1DYWEhAgMDIf5wdqtSqbBt2zZ8++23SElJQVZWFiIiIgAA8+bNQ2xsLADg0qVLuHTpUqNz\nTp48GQ4ODkhPT0dWVhaioqJgbm5+z20LCgowatQo9OnTB1lZWVi/fj22bNmChQsX6my3efNmGBsb\n49ChQ1i9ejU++eQTbNu27b45qqqq8PHHH2Pz5s3Ys2cPfvnlF/zzn//EV199hf/+979ITEzErl27\n8MUXX2jvU15ejjlz5iA9PR2pqalo27YtxowZg+rqagBAeno6ACA5ORmXLl1CQkKC9r5paWk4fvw4\nkpOTsWfPnjp53N3dsWrVKoSFhSE/Px/FxcUIDg5GREQEBg4c2LgntzUwdMO1VkeOHBEAREJCwn23\nS0lJEUZGRuLcuXPasfz8fKFQKMSPP/4ohLiz6jAzMxMlJSXabZYuXSrc3Ny0t7dv3y7+/NfdmBWM\njY2NiIuLu2e2P69g3n33XeHm5qazGtm4caMwNTUV5eXlQog7q4CBAwfq7GfEiBFi+vTp9T4HGzdu\nFADE6dOntWNz584VRkZG2lVafY/nj8rKyoSRkZE4cODAPfP/cT92dnaisrJSZ/yPK5i7AgMDxYAB\nA8QzzzwjnnzySaFWq+s9fmvEFYyBiEZeWz916hQcHR3x97//XTvm6uoKR0dH5ObmasdcXFzQtm1b\n7W1HR0dcuXLloXO+9dZbePXVV+Hv749ly5bh9OnT983q6+sLI6P//2Pl5+eH6upq7TUgAPDy8tK5\nX2OympmZwd3dXXu7Y8eO6NSpE+zs7HTG/rif/Px8TJ48GW5ubmjTpg06duwIjUbTqGtbHh4eMDMz\na3C7L7/8Enl5edi/fz/i4+OhVCobvE9rwoIxkK5du0KhUODUqVP33U4IUe9F2T+Om5iY1JnTaDT3\n3beRkVGdoqupqdG5HRUVhdzcXAQGBuLQoUPw8vLChg0b9J71z69SKRSKBvczZswYFBcXY+3atThy\n5AiysrJgbGysPUW6Hysrqwa3Ae68Onfz5k1UVlaioKCgUfdpTVgwBmJra4uAgACsXr0aZWVldeZL\nSkoAAD179kRBQQFUKpV27rfffkNhYSF69uz5UBns7e3rXI/Jzs6us13Xrl0xe/ZsfP/995g+fTq+\n/PLLe+6vZ8+eOHz4sM4v+cGDB2Fqago3N7eHyvqgrl27hlOnTuHdd9/FiBEj0KNHD5SWlkKtVmu3\nMTU1BXDn5fa/oqSkBC+99BLmzZuH0NBQvPjii7h161aT5G8pWDAG9MUXX0AIgX79+mH79u04c+YM\nTp8+jZiYGO1pxIgRI+Dt7Y0pU6bg6NGj+OWXXzBlyhT4+PjA39//oY7v7++PrKwsbNiwAXl5efjw\nww/x008/aedv376N0NBQpKamQqVS4ciRIzh48GC9xTZr1iwUFhZi1qxZOHXqFL7//nu88847CAsL\ng6Wl5UNlfVDt2rWDnZ0dYmNjkZeXh7S0NISEhOishDp06AALCwv873//Q1FREW7evPlAxwgJCYGd\nnR2WLFmCDz74ADY2NggNDW3qh9KssWAMqEuXLsjMzMRTTz2FBQsWwMvLC/7+/tixYwfWrl0L4M6y\nPzExEfb29hg6dCiGDRuGTp06ITEx8aHfzxIQEIDIyEhERESgb9++UKlUmDVrlnZeqVTixo0bmDZt\nGtzd3REUFARfX1+sXLnynvtzcnJCUlISsrKy0Lt3b7zyyiuYNGkSli9f/lA5/wojIyNs27YNx48f\nh4eHB0JDQ/Hee+/pXFcxNjbGZ599hi+//BKOjo547rnnGr3/TZs2YceOHdi8eTNMTExgZmaGr7/+\nGt988w22bNki4yE1S3wnLxFJwxUMEUnDgiEiaVgwRCQNC4aIpHkkPk1dWVmJnJwc2Nvb852QRM3M\n3a/t8PDwqPM5tUeiYHJycjBlyhRDxyCih7B58+Y636vzSBSMvb09gDsBW8P3i7RGvXp5NbwRNUtK\npRHs7dtrf4//6JEomLunRZ06dYKzs7OB05AMtbX3/6wRNX/3urzBi7xEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVD\nRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQk\nDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCaSL79+/H2LFj4eTkBIVCgbi4OO1cTU0NFixY\nAC8vL1hZWcHBwQGTJ0/G+fPntdtcv34d4eHh6N69OywsLNC5c2fMnDkT165dM8CjocYYNOhJbNu2\nBWfP5qKsrARTpkzWmbeyssJHH32IM2dOorj4EjIzMxAaOstAaQ2DBdNEysrK4OHhgU8//RQWFhY6\ncxUVFcjMzERERAQyMzPx3Xff4cKFC3j66aehVqsBAIWFhSgoKMCHH36IEydOID4+Hvv378ekSZMM\n8XCoEaysrJCbm4v5899BRUVFnfno6GUICBiJGTNC0LfvAKxY8TGWLInExIkTDJDWMBRCCGHoEBcv\nXsTw4cOxZ88eODs7GzrOQ7O2tsbq1asRHBxc7za5ubno1asXjh8/Dk9Pz3tu88MPP+DZZ59FSUkJ\n2rRpIymtflhbP2boCFJdvnwRc+fOx+bNX2vH0tMP4bvvdmLZsve1Y8nJ3+PkyZOYO3e+IWJKoVQa\noVMn+3v+/nIFYyC3bt0CALRr1+6+25iZmcHS0lJfsagJHT78M0aNehpOTk4AgAEDnoCnpwd+/HGP\ngZPpj7GhA7RG1dXVmDt3LsaMGVPviq2kpASLFi3CjBkzYGzMv6bmaN68Bfj001U4c+Ykampq/m9s\nPpKT/2fgZPrDn1w9U6vVmDp1KkpKSrBjx457blNeXo4xY8bAyckJH374oZ4TUlMJCXkdAwcOwLhx\nE3H+/AX4+T2JZcvew++/n8fu3a1jFSO1YOLj45GQkICzZ8/i2WefRXR0tMzDPfLUajUmTZqEEydO\nIDU1Fe3bt6+zTVlZGZ555hkAwK5du2Bubq7vmNQEzM3NsXjxv/Hii8FISkoGAJw8eRKenp54441w\nFkxT6NChA2bNmoUDBw6gqqpK5qEeeTU1NZg4cSJycnKQmpqKTp061dmmtLQUo0aNghACycnJsLa2\nNkBSagomJiYwNTVFbW2tzrhGUwsjo9Zz6VNqwYwcORIAcOLECRQVFck8lMGVlZUhLy8PAKDRaHD+\n/HlkZ2fD1tYWjo6OGDduHDIyMrBz504oFApcvnwZANC2bVtYWFigtLQUI0eOxK1bt5CYmIjy8nKU\nl5cDAGxtbWFqamqwx0b3ZmVlBVdXVwCAkZEROnd2hqenJ27cuIGLFy/iwIGDWLIkEuXl5f93ijQI\nkyZNxKJFkQZOrj96eZl61apVKCoqqvcUqSW8TJ2amophw4bVGZ82bRqioqLQpUuXe95v48aNCA4O\nrvf+ALBv3z4MHTq0KePqXUt8mXrwYD8kJe2qMx4f/zVCQmahQ4cOWLw4EsOHD0O7du1w4cIFxMX9\nB599ttoAaeW538vUvMjbRIYOHYr7dXVDPd7Q/enRc+DAwfsW55UrVzBzZqgeEz16Ws/JIBHpHQuG\niKSReoqkVqtRW1sLjUaD2tpaVFVVQalU8o1jRK2E1BVMTEwMvLy8sG7dOuzYsQNeXl6IiYmReUgi\neoRIXUqEh4cjPDxc5iGI6BHGazBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgw\nRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE\n0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGuP6Jm7dunXfO7Zp06bJ\nwxBRy1JvwfTq1QsKhQJCCO3Y3dsKhQLnz5/XS0Aiar7qLZgLFy7oMwcRtUCNugazdetWLF++HABw\n8eJFHD16VGooImoZGiyYsLAw7Nu3D5s2bQIAWFpaIiQkRHowImr+6j1FuuvQoUPIzMxEnz59AAC2\ntraorq6WHoyImr8GVzAmJibQaDRQKBQAgGvXrsHIiK9uE1HDGmyK0NBQvPDCCyguLkZkZCT8/Pyw\nYMECfWQjomauwVOkl156CX379sXu3bsBANu3b4eHh4f0YETU/DVYMABQW1sLExMTKBQKaDQa2ZmI\nqIVo8BRp2bJlmDRpEgoLC3Hx4kVMnjwZ77//vj6yEVEz1+AKJj4+HkePHoWlpSUAICIiAn379sXC\nhQulhyOi5q3BFYyLiwvUarX2tlqthqurq9RQRNQy1LuCefPNN6FQKGBpaYlevXohICAACoUCKSkp\n8PPz02dGImqm6i2Yu68U9erVC6NHj9aODxw4UH4qImoR6i2Y6dOn6zMHEbVADV7kzc/PR0REBHJz\nc1FZWakdP3v2rNRgRNT8NXiRNzg4GC+//DKEEEhKSsL48eMxceJEfWQjomauwYKpqKhAQEAAAMDN\nzQ1Lly7Fvn37pAcjouavwVMkMzMzCCHg5uaGNWvWwMnJCVeuXNFHNiJq5hosmFWrVqGsrAyfffYZ\nIiIicPPmTWzYsEEf2YiomWuwYAYMGAAAsLGx0X7pFBFRY9RbMEFBQdrvgLmXhIQEKYGIqOWot2DC\nwsL0mQMAEBz8LkxMzPV+XJKvvPymoSOQJMbGxgDs7z1X352GDx8uKw8RtRL87ksikoYFQ0TSNLpg\nqqqqZOYgohaowYJJT0+Hp6cnunbtCgA4duwYwsPDpQcjouavwYKZPXs2du3ahfbt2wMAvL29+VEB\nImqUBgtGo9HAxcVFZ0ypVEoLREQtR4Pv5O3cuTPS09OhUChQW1uLzz//HN26ddNHNiJq5hpcwcTE\nxGDlypU4f/48OnbsiJ9//hkxMTH6yEZEzVyDK5gOHTpg69at+shCRC1MgwUzY8aMe34mad26dVIC\nEVHL0WDBjBgxQvvnyspKfPvtt+jcubPUUETUMjRYMBMmTNC5/eKLL+Kpp56SFoiIWo4H/qjAuXPn\n8Pvvv8vIQkQtTIMrmHbt2mmvwWg0Gtja2iI6Olp6MCJq/u5bMEIIHDt2DE5OTgAAIyOj+34JFRHR\nH933FEmhUCAoKAhKpRJKpZLlQkQPpMFrME888QQyMzP1kYWIWph6T5HUajWMjY1x8OBBxMbGws3N\nDVZWVhBCQKFQsHSIqEH1FszdlUtiYqI+8xBRC1JvwQghANz51xyJiP6KegumuLgYK1eurPeOb731\nlpRARNRy1FswtbW1KCsr065kiIgeVL0F4+DggH//+9/6zEJELUy9L1Nz5UJED6vegtmzZ48+cxBR\nC1Rvwdja2uozBxG1QPyH14hIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoW\nDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQ\nkTQsGCKShgWjJxMmjEJS0jrMnDlJO2ZuboaZMydi06YPkJi4GrGxSxAYOMKAKelBDB48GN999x0u\nXrwIIQSmTZumnTM2NkZ0dDSOHTuGsrIyFBYWYvPmzejcubMBE+sfC0YPunfvgqefHozffrugM/7a\na+PQv78nVqzYgNdei8TWrT/glVeC4O8/0EBJ6UFYW1sjJycHb7zxBioqKnTmLC0t4ePjg2XLlsHH\nxwfPPfccOnfujOTkZCiVSgMl1j9jQwdo6SwtLTB//qv45JOvMHnyszpzPXq4Ye/en3H8+BkAwJ49\n1xAQ4Ad39y7Yu/dnQ8SlB5CUlISkpCQAQFxcnM7crVu3MHLkSJ2x119/Hbm5uejRowdycnL0FdOg\nuIKRbPbsqTh48CiOHTtTZ+7kyTwMGOANO7t2AIAePVzh6toZR4+2jh++1qZNmzYAgBs3bhg4if5w\nBSPR00/7wdGxAz76aMM959es2Yrw8KnYtOkDqNW1AICYmC1ITz+hz5ikByYmJvj444+xY8cOFBQU\nGDqO3rBgJHFy6ojg4CDMm7dCWx5/NnasP3r2dENU1GoUFV2Dp2c3vPrqP1FUdA1Hj57Uc2KSRalU\nIj4+Ho899hjGjh1r6Dh6Ja1gqqurERUVhcOHD6OkpAQuLi548803MWTIEFmHfKT06OGKtm1tsGZN\npHZMqVTCw6MrRo/+B8aPfwvBwUFYvnwtjhw5DgBQqQrg6uqMF14YyYJpIZRKJbZs2QJPT08MHToU\n169fN3QkvZJWMGq1Gg4ODti0aRMcHR2RlpaGOXPmYOfOnXB2dpZ12EfG4cPZCAmJ0hl7661gFBQU\nYdu2OxcGTUyModFodLbRaAQUCoW+YpJExsbG2Lp1Kzw8PDB06FAUFRUZOpLeSSsYS0tLhIeHa28P\nGzYMzs7OOHnyZKsomPLy2ygvv60zVllZhdLSCvz+eyEA4PjxM3j55edx+3YVrly5c4o0fPhAbNjw\nX0NEpgdkZWWFxx9/HABgZGSEv/3tb/D29sb169dRWFiI7du3o3///hgzZgyEEOjYsSMA4ObNm6is\nrDRkdL3R26tIV69ehUql0v6FEBAdHYuzZ1WYP3861q5djPHjR2HTpu+wY8c+Q0ejRujXrx+ys7OR\nnZ0NS0tLLFmyBNnZ2ViyZAmcnZ0RGBgIJycnZGZm4vLly9r/JkyYYOjoeqOXi7w1NTWYN28egoKC\n4Obmpo9DPpIWLPhY5/aNG7ewatVXBkpDDystLe2+p7M81dXDCkaj0WD+/PkwMTHBokWLZB+OiB4h\nUlcwQghERETg6tWriI2NhYmJiczDEdEjRmrBREZGIj8/Hxs3boS5ubnMQxHRI0hawRQUFGDbtm0w\nNTWFn5+fdnzx4sWt7s1GRK2VtIJxcnLCmTN1P39DRK0HP+xIRNKwYIhIGhYMEUnDgiEiaVgwRCQN\nC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBg\niEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aI\npGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEga\nFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaYwNHQAAamtrAQBq\ndZWBk5AsxsaPxI8aSXD37/bu77HOnL7D3EtxcTEA4NKlbAMnIVlcXV0NHYEkKy4uhouLi86YQggh\nDJRHq7KyEjk5ObC3t4dSqTR0HCJ6ALW1tSguLoaHhwfMzc115h6JgiGilokXeYlIGhYMEUnDgiEi\naVgwRCQNC0bP4uPj8fzzz8PDwwPvvPOOoeNQE6qursa7776LYcOGoU+fPggMDERaWpqhYxnUI/E+\nmNakQ4cOmDVrFg4cOICqKr6xsCVRq9VwcHDApk2b4OjoiLS0NMyZMwc7d+6Es7OzoeMZBAtGz0aO\nHAkAOHHiBIqKigychpqSpaUlwsPDtbeHDRsGZ2dnnDx5stUWDE+RiCS5evUqVCoVHn/8cUNHMRgW\nDJEENTU1mDdvHoKCguDm5mboOAbDgiFqYhqNBvPnz4eJiQkWLVpk6DgGxWswRE1ICIGIiAhcvXoV\nsbGxMDExMXQkg2LB6JlarUZtbS00Gg1qa2tRVVUFpVLJrzNoISIjI5Gfn4+NGzfW+eBfa8QPO+rZ\n559/jtWrV+uMhYWF6bz6QM1TQUEB/P39YWpqqvM/jMWLF2Ps2LEGTGY4LBgikoYXeYlIGhYMEUnD\ngiEiaVgwRCQNC4aIpGHBEJE0LJgWSqlUonfv3vDw8MC4ceNQUVHxl/eVmpqKZ599FgCwY8cOREdH\n17ttSUkJvvjiiwc+RlRUFD766KNGj/9RcHAwvvnmm0YfS6VSwcPD44Ez0oNjwbRQFhYWyM7ORk5O\nDkxNTbFmzRqdeSEENBrNA+937Nix9/2irL9aMNQysWBagcGDByMvLw8qlQo9evTArFmz4OPjgwsX\nLiAlJQW+vr7w8fHBuHHjUFZWBgBITk5G9+7d4efnh4SEBO2+4uLiEBYWBgAoKipCUFAQvL294e3t\njUOHDuGdd95Bfn4+evfujbfffhsAsGLFCvTv3x9eXl6IjIzU7mvZsmVwd3fHiBEjcObMmQYfR2xs\nLPr37w9vb2+88MILOquy3bt3Y/DgwejWrRt27doF4M6/1/P2229rj7127dqHfzLpgbBgWji1Wo2k\npCR4enoCAM6cOYOXXnoJWVlZsLKywtKlS7F7925kZmaiX79+WLlyJSorKzFjxgzs3LkTBw4cwOXL\nl++579mzZ2PIkCE4duwYMjMz0atXL0RHR8PNzQ3Z2dlYsWIFUlJS8OuvvyI9PR3Z2dk4evQo9u/f\nj6NHj2Lr1q3IyspCQkICMjIyGnwszz//PDIyMnDs2DH06NED69ev186pVCqkpaXh+++/R0hICCor\nK7F+/Xq0bdsWGRkZyMjIQGxsLM6dO9c0Tyw1Cj9h10Ldvn0bvXv3BnBnBTN9+nQUFhbCxcUFAwcO\nBAD8/PPPyM3NxaBBgwDc+U5ZX19fnD59Gl26dEHXrl0BAFOnTsW6devqHGPv3r34z3/+A+DONZ+2\nbdvixo0bOtukpKQgJSUFffr0AQCUlZXh119/RWlpKYKCgmBpaQkAjfqsTk5ODv71r3+hpKQEZWVl\nCAgI0M6NHz8eRkZG6Nq1K1xdXXH69GmkpKTg+PHj2uszN2/exK+//opu3bo1/omkh8KCaaHuXoP5\nMysrK+2fhRB46qmnsGXLFp1tsrOzoVAomiSHEAILFy7E66+/rjP+ySefPPAxgoODkZiYCG9vb8TF\nxSE1NVU79+d9KRQKCCHw+eef6xQRcGe1Q/rBU6RWbODAgfjpp5+Ql5cHAKioqMDZs2fRvXt3nDt3\nDvn5+QBQp4DuGj58OGJiYgDcud5x69Yt2NjYoLS0VLtNQEAANmzYoL22U1BQgCtXruAf//gHvv32\nW9y+fRulpaXYuXNng3lLS0vh4OCAmpoabN68WWdu+/bt0Gg0yM/Px2+//QZ3d3cEBAQgJiYGNTU1\nAICzZ8+ivLz8AZ8lehhcwbRi9vb2iIuLw6RJk7T/wsHSpUvRrVs3rFu3DqNHj4adnR38/PyQk5NT\n5/6ffvopXnvtNaxfvx5KpRIxMTHw9fXFoEGD4OHhgVGjRmHFihU4deoUfH19AQDW1taIj4+Hj48P\nJkyYgN69e8PFxQWDBw9uMO97772HAQMGwMXFBZ6enjpF5u7ujiFDhqCoqAhr1qyBubk5Xn31VahU\nKvj4+EAIAXt7eyQmJjbRs0eNwa9rICJpeIpERNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0\nLBgikub/Ac9zhqEMylSeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x103598410>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    2.0s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model ExtraTreesClassifier(bootstrap=False, class_weight=None, criterion='gini',\n",
      "           max_depth=None, max_features='auto', max_leaf_nodes=None,\n",
      "           min_impurity_split=1e-07, min_samples_leaf=1,\n",
      "           min_samples_split=2, min_weight_fraction_leaf=0.0,\n",
      "           n_estimators=10, n_jobs=1, oob_score=False, random_state=None,\n",
      "           verbose=0, warm_start=False)\n",
      "Grid Search Best Score: 0.715\n",
      "Grid Search Best Params: {'pca__n_components': 25}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.75      0.88      0.81       140\n",
      "          2       0.53      0.32      0.40        60\n",
      "\n",
      "avg / total       0.68      0.71      0.69       200\n",
      "\n",
      "Accuracy on Test Data: 0.710\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHetJREFUeJzt3XtcVGXiBvBnuMpNw0C5KSkimQhesGDFUKLISoNcy0up\nm9m6gq2lpUYFlZqbLd1cSUnBVVfNRPKyGElKlpUoKKGgQYxcNAQNAZHLMO/vD3/OOiGCyTtHhuf7\n+fD5OOc9c84zAzy+55yZQSWEECAiksBE6QBEZLxYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgOpGY\nmBj07NkTKpUKiYmJt7w9tVoNlUqFw4cP33q4DmDUqFGIjIxUOkaHouLrYJRVVlaGpUuXYteuXSgp\nKYGDgwN8fHwwZ84cPPLII+22n5ycHAwaNAhJSUkICAhAt27dYGVldUvbbGpqQnl5ORwcHGBmZtZO\nSQ1HrVajT58+yMjIgJ+fX6vrX7hwAebm5rCzszNAOuPQ8X4qjIharcaIESNgZ2eHd955B76+vtBq\ntUhLS8OsWbNQVFTUbvvKz88HAISFhUGlUrXLNk1NTeHk5NQu27qdNTQ0wMLCAt27d1c6SscjSDFj\nxowRzs7Oorq6utnYhQsXdP8+ffq0CAsLE7a2tsLW1laEh4eL4uJi3Xh0dLQYOHCg2LRpk+jbt6+w\ntbUVjz/+uCgvL9eNA9D7EkKIadOmiUcffVRvv1e3dVV2drYIDg4WdnZ2wtbWVvj4+Iivv/5aCCFE\nYWGhACAyMjJ066enp4t7771XWFpaih49eoi5c+eK+vp63XhQUJD429/+JhYtWiTuvPNO4ejoKObN\nmyeamppafJ4SEhKEjY2N+O9//yu8vLyElZWVGDt2rKisrBRbt24V/fr1E127dhVPP/20qK2t1d0v\nJSVFBAYGijvuuEPY29uLhx56SJw4cUI3/vvnJCgoSO95WbZsmXB1dRWOjo667BEREUIIIfLy8oS1\ntbVYt26d3v7Mzc3F999/3+Jj6Wx4DkYhFy5cwJ49exAZGQlbW9tm4/b29gAAIQTCwsJQVlaGr7/+\nGvv27cOZM2cQFhYGcc3RrVqtxpYtW7B9+3akpqYiKysLUVFRAID58+cjPj4eAHD27FmcPXu2zTkn\nT54MZ2dnHDp0CFlZWYiJiUGXLl2uu25paSnGjBmDIUOGICsrC2vWrMGmTZuwaNEivfU2btwIMzMz\nHDx4ECtWrMAHH3yALVu23DBHfX09/vnPf2Ljxo1IS0vD4cOH8ec//xnr1q3Dtm3bkJycjF27dmHl\nypW6+1y6dAlz587FoUOHsH//fnTr1g1jx45FQ0MDAODQoUMAgD179uDs2bNISkrS3Tc9PR3Z2dnY\ns2cP0tLSmuXx8vLC+++/j8jISBQUFKC8vBzTp09HVFQU/P392/bkdgZKN1xn9eOPPwoAIikp6Ybr\npaamChMTE1FYWKhbVlBQIFQqlfjqq6+EEFdmHZaWlqKyslK3zuLFi4WHh4fu9tatW8Xvv91tmcHY\n2dmJxMTE62b7/Qzm1VdfFR4eHnqzkYSEBGFhYSEuXbokhLgyC/D399fbTkhIiJgxY0aLz0FCQoIA\nIPLy8nTL5s2bJ0xMTHSztJYez7VqamqEiYmJOHDgwHXzX7sdBwcHUVdXp7f82hnMVWFhYeK+++4T\njzzyiPjTn/4kNBpNi/vvjDiDUYho47n13NxcuLi44K677tIt69u3L1xcXHDixAndMnd3d3Tr1k13\n28XFBefOnbvlnC+99BKee+45BAcHY8mSJcjLy7th1oCAAJiY/O/HKjAwEA0NDbpzQADg4+Ojd7+2\nZLW0tISXl5fuds+ePeHk5AQHBwe9Zddup6CgAJMnT4aHhwe6du2Knj17QqvVtunclre3NywtLVtd\n79NPP0V+fj6++eYbbNiwAaampq3epzNhwSjE09MTKpUKubm5N1xPCNHiSdlrl5ubmzcb02q1N9y2\niYlJs6JrbGzUux0TE4MTJ04gLCwMBw8ehI+PD9auXWvwrL+/SqVSqVrdztixY1FeXo5Vq1bhxx9/\nRFZWFszMzHSHSDdiY2PT6jrAlatzFy9eRF1dHUpLS9t0n86EBaOQ7t27IzQ0FCtWrEBNTU2z8crK\nSgDAPffcg9LSUqjVat3YL7/8gjNnzuCee+65pQyOjo7NzsccPXq02Xqenp544YUXsHv3bsyYMQOf\nfvrpdbd3zz334Pvvv9f7Jf/2229hYWEBDw+PW8p6s86fP4/c3Fy8+uqrCAkJwYABA1BdXQ2NRqNb\nx8LCAsCVy+1/RGVlJaZOnYr58+cjIiICzzzzDKqqqtolv7FgwSho5cqVEELAz88PW7duxcmTJ5GX\nl4e4uDjdYURISAh8fX0xZcoUHDlyBIcPH8aUKVMwdOhQBAcH39L+g4ODkZWVhbVr1yI/Px/vvvsu\nvvvuO9345cuXERERgf3790OtVuPHH3/Et99+22KxzZ49G2fOnMHs2bORm5uL3bt3Y+HChYiMjIS1\ntfUtZb1Z9vb2cHBwQHx8PPLz85Geno5Zs2bpzYR69OgBKysrfPnllygrK8PFixdvah+zZs2Cg4MD\n3nrrLfzjH/+AnZ0dIiIi2vuhdGgsGAX16dMHmZmZePDBB7FgwQL4+PggODgYO3bswKpVqwBcmfYn\nJyfD0dERo0aNwujRo+Hk5ITk5ORbfj1LaGgooqOjERUVhWHDhkGtVmP27Nm6cVNTU/z222+YNm0a\nvLy8EB4ejoCAAMTGxl53e66urkhJSUFWVhYGDx6MZ599FpMmTcLSpUtvKecfYWJigi1btiA7Oxve\n3t6IiIjA22+/rXdexczMDB999BE+/fRTuLi44PHHH2/z9tevX48dO3Zg48aNMDc3h6WlJf7zn//g\n888/x6ZNm2Q8pA6Jr+QlImk4gyEiaVgwRCQNC4aIpGHBEJE0t8W7qevq6pCTkwNHR0e+EpKog7n6\nsR3e3t7N3qd2WxRMTk4OpkyZonQMIroFGzdubPa5OrdFwTg6OgK4ErAzfL5IZ9SnTx+lI5AkZmZm\n6N27t+73WG9MgTzNXD0scnJygpubm8JpSIZrX6JPxul6pzd4kpeIpGHBEJE0LBgikoYFQ0TSsGCI\nSBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0Lhoik\nYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoW\nDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQ\nkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgw7eSbb77BuHHj4OrqCpVKhcTERN1YY2MjFixYAB8f\nH9jY2MDZ2RmTJ09GUVGR3jZmzpwJDw8PWFlZwdHREY8//jhyc3MN/EioLUaOHIkvvvgCJSUlEEJg\n2rRpeuNCiOt+rVixQqHEymDBtJOamhp4e3vjww8/hJWVld5YbW0tMjMzERUVhczMTHzxxRcoLi7G\nww8/DI1Go1vPz88PiYmJyM3NxZdffgkhBEJCQtDY2Gjoh0OtsLW1RU5ODv7+97+jtra22biTk5Pe\n12OPPQYA+OyzzwwdVVniNlBcXCz69+8viouLlY7SLmxsbERCQsIN1zl+/LgAILKzs1tc59ixYwKA\nyMvLa+eEhgfAaL+qq6vFtGnTbrjO6tWrRV5enuJZZXyZmZm1+PvLGYxCqqqqAAD29vbXHb906RIS\nEhLQu3dv3HXXXQZMRu3NxsYGEydORHx8vNJRDI4Fo4CGhgbMmzcPY8eOhZubm97YypUrYWtrC1tb\nW6SkpCAtLQ2WlpYKJaX2MHnyZFhaWmLdunVKRzE4FoyBaTQaPP3006isrERCQkKz8SlTpiArKwvp\n6eno378/JkyYcN1jfOo4Zs6cieTkZFRUVCgdxeCkFsyGDRvwxBNPwNvbGwsXLpS5qw5Bo9Fg0qRJ\nyM7ORlpaGu68885m63Tr1g2enp64//778fnnn+PUqVPYtm2bAmmpPfj6+mL48OGd8vAIAMxkbrxH\njx6YPXs2Dhw4gPr6epm7uu01NjZi4sSJyMnJwf79++Hk5NTqfcT/X9rs7M9dR/b888+jsLAQe/fu\nVTqKIqQWzEMPPQQA+Omnn1BWViZzV4qrqalBfn4+AECr1aKoqAhHjx5F9+7d4eLiggkTJiAjIwM7\nd+6ESqXCr7/+CuDKjMXKygr5+fnYtm0bQkJC4OjoiJKSEixbtgyWlpa6S5x0+7CxsUG/fv0AACYm\nJujduzd8fX1x4cIFFBcXAwCsrKwwZcoUvPvuu0pGVZYhLlHGxsaKBQsWtDhuDJep9+3bd91LeNOm\nTROFhYUtXuK7ejm7qKhIPPzww8LR0VGYm5sLNzc3MXnyZJGbm6vsA2snLT3+jvoVFBR03ceZkJCg\nW2f69OmisbFRODs7K55X5teNLlNLncF0JqNGjcKV36Pru9EYAPTq1QspKSntHYskSU9Ph0qluuE6\niYmJeq/o7ox4FYmIpGHBEJE0Ug+RNBoNmpqaoNVq0dTUhPr6epiamsLMjEdmRJ2B1BlMXFwcfHx8\nsHr1auzYsQM+Pj6Ii4uTuUsiuo1InUrMmTMHc+bMkbkLIrqN8RwMEUnDgiEiaVgwRCQNC4aIpGHB\nEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwR\nScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0\nLBgiksaspYGqqqob3rFr167tHoaIjEuLBTNw4ECoVCoIIXTLrt5WqVQoKioySEAi6rhaLJji4mJD\n5iAiI9SmczCbN2/G0qVLAQAlJSU4cuSI1FBEZBxaLZjIyEjs27cP69evBwBYW1tj1qxZ0oMRUcfX\n4iHSVQcPHkRmZiaGDBkCAOjevTsaGhqkByOijq/VGYy5uTm0Wi1UKhUA4Pz58zAx4dVtImpdq00R\nERGB8ePHo7y8HNHR0QgMDMSCBQsMkY2IOrhWD5GmTp2KYcOGYe/evQCArVu3wtvbW3owIur4Wi0Y\nAGhqaoK5uTlUKhW0Wq3sTERkJFo9RFqyZAkmTZqEM2fOoKSkBJMnT8Y777xjiGxE1MG1OoPZsGED\njhw5AmtrawBAVFQUhg0bhkWLFkkPR0QdW6szGHd3d2g0Gt1tjUaDvn37Sg1FRMahxRnMiy++CJVK\nBWtrawwcOBChoaFQqVRITU1FYGCgITMSUQfVYsFcvVI0cOBAPProo7rl/v7+8lMRkVFosWBmzJhh\nyBxEZIRaPclbUFCAqKgonDhxAnV1dbrlp06dkhqMiDq+Vk/yTp8+HX/5y18ghEBKSgqefPJJTJw4\n0RDZiKiDa7VgamtrERoaCgDw8PDA4sWLsW/fPunBiKjja/UQydLSEkIIeHh44JNPPoGrqyvOnTtn\niGxE1MG1WjDvv/8+ampq8NFHHyEqKgoXL17E2rVrDZGNiDq4VgvmvvvuAwDY2dnpPnSKiKgtWiyY\n8PBw3WfAXE9SUpKUQERkPFosmMjISEPmAAA89thUmJi06Q3e1MGYmvL7aqxu9L1tceSBBx6QEoaI\nOg9+9iURScOCISJp2lww9fX1MnMQkRFqtWAOHTqEQYMGwdPTEwBw7NgxzJkzR3owIur4Wi2YF154\nAbt27cKdd94JAPD19eVbBYioTVotGK1WC3d3d71lpqam0gIRkfFo9cUJvXr1wqFDh6BSqdDU1ISP\nP/4Y/fv3N0Q2IurgWp3BxMXFITY2FkVFRejZsyd++OEHxMXFGSIbEXVwrc5gevTogc2bNxsiCxEZ\nmVYLZubMmdd9T9Lq1aulBCIi49FqwYSEhOj+XVdXh+3bt6NXr15SQxGRcWi1YJ566im928888wwe\nfPBBaYGIyHjc9FsFCgsLcfr0aRlZiMjItDqDsbe3152D0Wq16N69O5YtWyY9GBF1fDcsGCEEjh07\nBldXVwCAiYnJDT+EiojoWjc8RFKpVAgPD4epqSlMTU1ZLkR0U1o9B3PvvfciMzPTEFmIyMi0eIik\n0WhgZmaGb7/9FvHx8fDw8ICNjQ2EEFCpVCwdImpViwVzdeaSnJxsyDxEZERaLBghBIArf82RiOiP\naLFgysvLERsb2+IdX3rpJSmBiMh4tFgwTU1NqKmp0c1kiIhuVosF4+zsjDfeeMOQWYjIyLR4mZoz\nFyK6VS0WTFpamiFzEJERarFgunfvbsgcRGSE+IfXiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQs\nGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4Ih\nImlYMEQkDQuGiKRhwRCRNCwYIpKGBWMgzz47EZmZqViwIEK3LDh4BP71r6VIS/sMmZmpGDbMR8GE\ndLNGjgzE9u1JOH1aDY2mEVOnTtUb79GjB9asWYOiotOoqrqI3bt3oV+/fgqlVQYLxgAGDbob4eGP\n4NSpAr3lVlZdcOzYCcTGrlIoGd0KW1tbHD9+HC+++BJqa2ubjSclbYOnZz+MH/9n+PkNx+nTRfjy\nyz2wtrZWIK0yzJQOYOxsba2xePFCvPVWLGbOnKI3tnv3lT/Pe8cdXZWIRrcoJWUPUlL2AADWrl2j\nN+bp6Ql/f38MHToM2dnZAICIiAiUlpZg4sSJWLt2rcHzKoEzGMlee20u0tIOICPjqNJRyIAsLS0B\nAHV1dbplQgjU19djxIgRSsUyOBaMROHhY9CrlytWrlyndBQysLy8PKjVaixe/Dbs7e1hbm6Ol1+e\nj169esHZ2UnpeAbDgpHE3d0NkZF/QVTUMmg0GqXjkIFpNBo8+eRT6NvXA+Xl51BdXYVRo0YhJSUF\nTU1NSsczGGnnYBoaGhATE4Pvv/8elZWVcHd3x4svvoigoCBZu7yt+PgMgL39Hfjss9W6ZWZmphg6\ndBDGj38MI0aMQ2Njo4IJSbbMzEz4+fmha9eusLCwQEVFBQ4e/A6HDx9ROprBSCsYjUYDZ2dnrF+/\nHi4uLkhPT8fcuXOxc+dOuLm5ydrtbWPfvoM4ceJ5vWUxMfNQVFSKtWs3s1w6kaqqKgBAv379MGzY\nMERHRyucyHCkFYy1tTXmzJmjuz169Gi4ubnh+PHjnaJgamouoabmkt6yy5frUFVVjYICNQCga1c7\nODk5ws7OFgDQq5cLqqtrcP78bzh//jdDR6abZGNjo3tdi4mJCXr37gVfX19cuHABxcXFGD9+PM6f\nr8Dp00Xw9vbG++/H4osvvsBXX+1VOLnhGOwydUVFBdRqdad7odGNBAX54803X9bdfuONlwAAq1at\nx6pV65WKRW3k5zcMaWlputsxMTGIiYnBunX/xowZM+Ds7Iz33luOnj174uzZs9iwYQMWL16iYGLD\nUwkhhOydNDY2YubMmejduzfeeuutZuMlJSV44IEHYGXlChMTvjTHGGVn71M6AkliZmYGd/feSEtL\na3Z0Iv0qklarxSuvvAJzc3O8/vrrsndHRLcRqdMFIQSioqJQUVGB+Ph4mJuby9wdEd1mpBZMdHQ0\nCgoKkJCQgC5dusjcFRHdhqQVTGlpKbZs2QILCwsEBgbqlr/55psYN26crN0S0W1EWsG4urri5MmT\nsjZPRB0A3ypARNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJ\nw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQs\nGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4Ih\nImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiacyUDgAATU1NAACtVqNwEpLFzOy2+FEjCUxNTQH87/f4WrfFd728\nvBwAUF9fpnASksXdvbfSEUiy8vJyuLu76y1TCSGEQnl06urqkJOTA0dHR10bElHH0NTUhPLycnh7\ne6NLly56Y7dFwRCRceJJXiKShgVDRNKwYIhIGhYMEUnDgjGwDRs24IknnoC3tzcWLlyodBxqRw0N\nDXj11VcxevRoDBkyBGFhYUhPT1c6lqJui9fBdCY9evTA7NmzceDAAdTX1ysdh9qRRqOBs7Mz1q9f\nDxcXF6Snp2Pu3LnYuXMn3NzclI6nCBaMgT300EMAgJ9++gllZXxhoTGxtrbGnDlzdLdHjx4NNzc3\nHD9+vNMWDA+RiCSpqKiAWq1Gv379lI6iGBYMkQSNjY2YP38+wsPD4eHhoXQcxbBgiNqZVqvFK6+8\nAnNzc7z++utKx1EUz8EQtSMhBKKiolBRUYH4+HiYm5srHUlRLBgD02g0aGpqglarRVNTE+rr62Fq\nasqPMzAS0dHRKCgoQEJCQrM3/nVGfLOjgX388cdYsWKF3rLIyEi9qw/UMZWWliI4OBgWFhZ6/2G8\n+eabGDdunILJlMOCISJpeJKXiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCMVKmpqYYPHgwvL29\nMWHCBNTW1v7hbe3fvx+PPfYYAGDHjh1YtmxZi+tWVlZi5cqVN72PmJgYvPfee21efq3p06fj888/\nb/O+1Go1vL29bzoj3TwWjJGysrLC0aNHkZOTAwsLC3zyySd640IIaLXam97uuHHjbvhBWX+0YMg4\nsWA6gZEjRyI/Px9qtRoDBgzA7NmzMXToUBQXFyM1NRUBAQEYOnQoJkyYgJqaGgDAnj17cPfddyMw\nMBBJSUm6bSUmJiIyMhIAUFZWhvDwcPj6+sLX1xcHDx7EwoULUVBQgMGDB+Pll18GACxfvhzDhw+H\nj48PoqOjddtasmQJvLy8EBISgpMnT7b6OOLj4zF8+HD4+vpi/PjxerOyvXv3YuTIkejfvz927doF\n4Mrf63n55Zd1+161atWtP5l0U1gwRk6j0SAlJQWDBg0CAJw8eRJTp05FVlYWbGxssHjxYuzduxeZ\nmZnw8/NDbGws6urqMHPmTOzcuRMHDhzAr7/+et1tv/DCCwgKCsKxY8eQmZmJgQMHYtmyZfDw8MDR\no0exfPlypKam4ueff8ahQ4dw9OhRHDlyBN988w2OHDmCzZs3IysrC0lJScjIyGj1sTzxxBPIyMjA\nsWPHMGDAAKxZs0Y3plarkZ6ejt27d2PWrFmoq6vDmjVr0K1bN2RkZCAjIwPx8fEoLCxsnyeW2oTv\nsDNSly9fxuDBgwFcmcHMmDEDZ86cgbu7O/z9/QEAP/zwA06cOIERI0YAuPKZsgEBAcjLy0OfPn3g\n6ekJAHj66aexevXqZvv4+uuv8e9//xvAlXM+3bp1w2+//aa3TmpqKlJTUzFkyBAAQE1NDX7++WdU\nV1cjPDwc1tbWANCm9+rk5OTgtddeQ2VlJWpqahAaGqobe/LJJ2FiYgJPT0/07dsXeXl5SE1NRXZ2\ntu78zMWLF/Hzzz+jf//+bX8i6ZawYIzU1XMwv2djY6P7txACDz74IDZt2qS3ztGjR6FSqdolhxAC\nixYtwl//+le95R988MFN72P69OlITk6Gr68vEhMTsX//ft3Y77elUqkghMDHH3+sV0TAldkOGQYP\nkToxf39/fPfdd8jPzwcA1NbW4tSpU7j77rtRWFiIgoICAGhWQFc98MADiIuLA3DlfEdVVRXs7OxQ\nXV2tWyc0NBRr167VndspLS3FuXPncP/992P79u24fPkyqqursXPnzlbzVldXw9nZGY2Njdi4caPe\n2NatW6HValFQUIBffvkFXl5eCA0NRVxcHBobGwEAp06dwqVLl27yWaJbwRlMJ+bo6IjExERMmjRJ\n9xcOFi9ejP79+2P16tV49NFH4eDggMDAQOTk5DS7/4cffojnn38ea9asgampKeLi4hAQEIARI0bA\n29sbY8aMwfLly5Gbm4uAgAAAgK2tLTZs2IChQ4fiqaeewuDBg+Hu7o6RI0e2mvftt9/GfffdB3d3\ndwwaNEivyLy8vBAUFISysjJ88skn6NKlC5577jmo1WoMHToUQgg4OjoiOTm5nZ49agt+XAMRScND\nJCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0/wfQiJZkqDJgUAAAAABJRU5ErkJg\ngg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11540fe10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    0.5s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model SGDClassifier(alpha=0.0001, average=False, class_weight=None, epsilon=0.1,\n",
      "       eta0=0.0, fit_intercept=True, l1_ratio=0.15,\n",
      "       learning_rate='optimal', loss='hinge', n_iter=5, n_jobs=1,\n",
      "       penalty='l2', power_t=0.5, random_state=None, shuffle=True,\n",
      "       verbose=0, warm_start=False)\n",
      "Grid Search Best Score: 0.69\n",
      "Grid Search Best Params: {'pca__n_components': 25}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.74      0.74      0.74       140\n",
      "          2       0.40      0.40      0.40        60\n",
      "\n",
      "avg / total       0.64      0.64      0.64       200\n",
      "\n",
      "Accuracy on Test Data: 0.640\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH55JREFUeJzt3X9czffiB/DXqU4pxWX9UNpahYxUjKnJj8Lya1bc/SA2\nG7uzyr42ZuhuYZitK4yv0KhdGi6XLjbWJMnYiopFRekMRbIVJaU67+8fvs51Rsp4n9OP1/Px8Hj0\n+XE+n1dHvXp/3ueXQgghQEQkgYG+AxBR88WCISJpWDBEJA0LhoikYcEQkTQsGCKShgXTgsybNw82\nNjZQKBSIiYl55OOpVCooFAocO3bs0cM1AYMGDUJISIi+YzQpCj4PRr+KioqwePFi7NmzBxcvXoSl\npSXc3Nwwbdo0jBgx4rGdJzMzEz169MCOHTvg5eWFtm3bwtTU9JGOWVtbi+LiYlhaWsLIyOgxJdUd\nlUoFR0dHpKamonfv3vXu//vvv0OpVMLCwkIH6ZqHpvdT0YyoVCr069cPFhYW+Oyzz+Du7g61Wo2E\nhARMnToV58+ff2znys3NBQD4+/tDoVA8lmMaGhqiQ4cOj+VYjdmtW7dgbGyM9u3b6ztK0yNIb4YP\nHy5sbW1FWVnZPdt+//13zde//vqr8Pf3F+bm5sLc3FwEBASICxcuaLaHhYWJ7t27i82bNwsnJydh\nbm4uXnrpJVFcXKzZDkDrnxBCvPHGG2LkyJFa571zrDtOnjwpfH19hYWFhTA3Nxdubm7iwIEDQggh\n8vPzBQCRmpqq2T8pKUk899xzwsTERFhbW4vp06eLqqoqzfaBAweKd999V8yZM0c88cQTwsrKSsyY\nMUPU1tbWeT9FR0eL1q1bi++++064uLgIU1NT8eKLL4rS0lKxbds20alTJ9GmTRsxYcIEUVFRobnd\n3r17hbe3t/jLX/4i2rVrJ1544QVx+vRpzfY/3icDBw7Uul+WLFkiOnbsKKysrDTZg4ODhRBCZGdn\nCzMzM/H1119rnU+pVIqjR4/W+b20NJyD0ZPff/8d+/btQ0hICMzNze/Z3q5dOwCAEAL+/v4oKirC\ngQMHkJiYiMLCQvj7+0PcdXWrUqmwdetW7Ny5E/Hx8UhPT0doaCgAYObMmYiKigIAXLp0CZcuXWpw\nzvHjx8PW1hYpKSlIT0/HvHnz0KpVq/vuW1BQgOHDh6Nnz55IT0/H+vXrsXnzZsyZM0drv9jYWBgZ\nGeHIkSNYtWoVli9fjq1btz4wR1VVFZYuXYrY2FgkJCTg2LFj+Otf/4qvv/4a//73vxEXF4c9e/Zg\n9erVmtvcuHED06dPR0pKCg4ePIi2bdvixRdfxK1btwAAKSkpAIB9+/bh0qVL2LFjh+a2SUlJOHny\nJPbt24eEhIR78ri4uGDZsmUICQlBXl4eiouLMWnSJISGhsLT07Nhd25LoO+Ga6l+/vlnAUDs2LHj\ngfvFx8cLAwMDkZ+fr1mXl5cnFAqF+OGHH4QQt0cdJiYmorS0VLPPwoULhbOzs2Z527Zt4o//3Q0Z\nwVhYWIiYmJj7ZvvjCGbu3LnC2dlZazQSHR0tjI2NxY0bN4QQt0cBnp6eWscZMmSImDx5cp33QXR0\ntAAgsrOzNetmzJghDAwMNKO0ur6fu5WXlwsDAwORnJx83/x3H8fS0lJUVlZqrb97BHOHv7+/6Nu3\nrxgxYoR4/vnnRU1NTZ3nb4k4gtET0cC59aysLNjZ2eHpp5/WrHNycoKdnR1Onz6tWefg4IC2bdtq\nlu3s7HDlypVHzvnBBx9gypQp8PX1xaJFi5Cdnf3ArF5eXjAw+O+Plbe3N27duqWZAwIANzc3rds1\nJKuJiQlcXFw0yzY2NujQoQMsLS211t19nLy8PIwfPx7Ozs5o06YNbGxsoFarGzS35erqChMTk3r3\n++qrr5Cbm4tDhw5h06ZNMDQ0rPc2LQkLRk86d+4MhUKBrKysB+4nhKhzUvbu9Uql8p5tarX6gcc2\nMDC4p+iqq6u1lufNm4fTp0/D398fR44cgZubGzZs2KDzrH98lEqhUNR7nBdffBHFxcVYu3Ytfv75\nZ6Snp8PIyEhzifQgrVu3rncf4Pajc9euXUNlZSUKCgoadJuWhAWjJ+3bt4efnx9WrVqF8vLye7aX\nlpYCALp164aCggKoVCrNtnPnzqGwsBDdunV7pAxWVlb3zMdkZGTcs1/nzp3x3nvv4dtvv8XkyZPx\n1Vdf3fd43bp1w9GjR7V+yQ8fPgxjY2M4Ozs/UtaH9dtvvyErKwtz587FkCFD8Mwzz6CsrAw1NTWa\nfYyNjQHcfrj9zygtLcXrr7+OmTNnIjg4GBMnTsT169cfS/7mggWjR6tXr4YQAr1798a2bduQk5OD\n7OxsREZGai4jhgwZAnd3dwQGBuL48eM4duwYAgMD0atXL/j6+j7S+X19fZGeno4NGzYgNzcXX3zx\nBX788UfN9ps3byI4OBgHDx6ESqXCzz//jMOHD9dZbEFBQSgsLERQUBCysrLw7bffYvbs2QgJCYGZ\nmdkjZX1Y7dq1g6WlJaKiopCbm4ukpCRMnTpVayRkbW0NU1NTfP/99ygqKsK1a9ce6hxTp06FpaUl\nFixYgM8//xwWFhYIDg5+3N9Kk8aC0SNHR0ekpaVh6NCh+Oijj+Dm5gZfX1/s2rULa9euBXB72B8X\nFwcrKysMGjQIPj4+6NChA+Li4h75+Sx+fn4ICwtDaGgonn32WahUKgQFBWm2GxoaoqSkBG+88QZc\nXFwQEBAALy8vRERE3Pd4HTt2xN69e5Geng4PDw+89dZbGDduHBYvXvxIOf8MAwMDbN26FSdPnoSr\nqyuCg4Px6aefas2rGBkZ4csvv8RXX30FOzs7vPTSSw0+/saNG7Fr1y7ExsZCqVTCxMQE33zzDbZv\n347NmzfL+JaaJD6Tl4ik4QiGiKRhwRCRNCwYIpKGBUNE0jSKV1NXVlYiMzMTVlZWfCYkURNz5207\nXF1d73mdWqMomMzMTAQGBuo7BhE9gtjY2HveV6dRFIyVlRWA2wFbwvuLtEQDBgzXdwSSRkCprNb8\nHt+tURTMncuiDh06wN7eXs9pSA5O9zVft18acr/pDf6vE5E0LBgikoYFQ0TSsGCISBoWDBFJw4Ih\nImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVD\nRNKwYIhIGhYMEUnDgiEiaVgwRCQNC+YxOnToEEaPHo2OHTtCoVAgJiZGa7sQAvPmzYOdnR1MTU0x\naNAgnDp16r7HqqyshLu7OxQKBY4dO6aD9PQwJk58Fd99tw0nTvyIEyd+xPbt/4SPT3+tfRwdHRAZ\nGYGMjGScOvUTdu3aAmdnRz0l1g8WzGNUXl4OV1dXrFixAqampvds/+KLL7B06VKsXLkSqampsLa2\nxtChQ1FWVnbPvjNnzoS9vb0uYtOfcOlSET7/fDlGj34N/v7jcfRoCtasWYauXTsDAOztO+Jf/4rB\nhQsFCAx8G8OGjUVExCpUVFToObluGek7QHMyYsQIjBgxAgAwadIkrW1CCCxfvhyzZ8/G2LFjAQBf\nf/01rK2t8c033+Cdd97R7Puf//wHiYmJ2L59O7777jud5aeG27//oNby0qWrEBj4Cnr2dEd29lnM\nnBmCw4ePYvHipZp9Llwo0HFK/eMIRkfy8/Nx+fJlvPDCC5p1pqamGDBgAI4cOaJZd/HiRbz77ruI\njY297yiIGh8DAwOMGjUMZmZmSEvLgEKhgK/vQJw9ew7R0auRmpqIuLhYjBzpp++oOscRjI5cvnwZ\nAGBjY6O13sbGBgUFt/+y1dbWIjAwEDNmzICHhwdUKpWuY9JDcHHphO3bN8LExBgVFRV49933kZOT\nC0vLJ2Bu3hpBQVOwbNn/4osvVuD555/DsmWLcfPmTRw4cEjf0XWGBaNjCoVCa1kIoVm3ePFiKJVK\nfPDBB/qIRg/p3DkVRo16BW3aWGDYsCEID/8U48dPQWnpNQDA/v2JWL9+IwAgKysHPXp0w4QJr7ao\ngpF6ibRp0yaMGTMGrq6umD17tsxTNXodOnQA8N+RzB1XrlzRjGoSEhKQmJgIpVIJIyMjdOrUCQDg\n6emJwMBA3QamelVX1+DXXy/gl19OIzz8S2Rl5eCttyagpKQE1dXVOHv2nNb+ubn5sLOz1VNa/ZA6\ngrG2tkZQUBCSk5NRVVUl81SNnqOjIzp06IAffvgBffr0AXD7oejk5GSEh4cDAKKjo3Hjxg3NbQoL\nC+Hn54fY2Fj069dPL7mp4RQKAxgbG6O6ugYnT56Ck9PTWtsdHR1QWFion3B6IrVg7kxo/vLLLygq\nKpJ5qkahvLwcubm5AAC1Wo3z588jIyMD7du3x1NPPYXp06dj0aJF6Nq1K7p06YKFCxfC3Nwc48eP\nB3C7hO5mbm4OAHB2duZD1o3MrFn/g8TEQygsLIK5uRlGjx4BT8/emDw5BACwbl0MVq4MR2pqGo4e\nTYGnZx+MGuWHqVPf13Ny3eIczGN07Ngx+Pj4aJbDwsIQFhaGN954AzExMZg1axZu3ryJ4OBglJSU\noG/fvoiPj4eFhYUeU9OfYWn5BCIiFsPS0hJlZeXIyTmDN98MRnLy7UcEf/ghEaGhCxAUNAWffDIL\nKtV5zJz5dyQmJus5uW4phBBC9kmWLVuGoqIiLFmy5L7bL168iMGDByMhIYF/qZspJyd3fUcgadRQ\nKm/d9/eXz4MhImlYMEQkjdQ5mJqaGtTW1kKtVqO2thZVVVUwNDSEkRGnfohaAqkjmMjISLi5uWHd\nunXYtWsX3NzcEBkZKfOURNSISB1KTJs2DdOmTZN5CiJqxDgHQ0TSsGCISBoWDBFJw4IhImlYMEQk\nDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKw\nYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuG\niKQxqmvD9evXH3jDNm3aPPYwRNS81Fkw3bt3h0KhgBBCs+7OskKhwPnz53USkIiarjoL5sKFC7rM\nQUTNUIPmYLZs2YLFixcDAC5evIjjx49LDUVEzUO9BRMSEoLExERs3LgRAGBmZoapU6dKD0ZETV+d\nl0h3HDlyBGlpaejZsycAoH379rh165b0YETU9NU7glEqlVCr1VAoFACA3377DQYGfHSbiOpXb1ME\nBwdj7NixKC4uRlhYGLy9vfHRRx/pIhsRNXH1XiK9/vrrePbZZ7F//34AwLZt2+Dq6io9GBE1ffUW\nDADU1tZCqVRCoVBArVbLzkREzUS9l0iLFi3CuHHjUFhYiIsXL2L8+PH47LPPdJGNiJq4ekcwmzZt\nwvHjx2FmZgYACA0NxbPPPos5c+ZID0dETVu9IxgHBwfU1NRolmtqauDk5CQ1FBE1D3WOYN5//30o\nFAqYmZmhe/fu8PPzg0KhQHx8PLy9vXWZkYiaqDoL5s4jRd27d8fIkSM16z09PeWnIqJmoc6CmTx5\nsi5zEFEzVO8kb15eHkJDQ3H69GlUVlZq1p85c0ZqMCJq+uqd5J00aRLefPNNCCGwd+9evPLKK3jt\ntdd0kY2Imrh6C6aiogJ+fn4AAGdnZyxcuBCJiYnSgxFR01fvJZKJiQmEEHB2dsaaNWvQsWNHXLly\nRRfZiKiJq7dgli1bhvLycnz55ZcIDQ3FtWvXsGHDBl1kI6Imrt6C6du3LwDAwsJC86ZTREQNUWfB\nBAQEaN4D5n527NghJRARNR91FkxISIgucwAABgwYDn5UU/OUn39S3xFIEiMjozpfPlRnwQwePFha\nICJqGThcICJpWDBEJE2DC6aqqkpmDiJqhuotmJSUFPTo0QOdO3cGAJw4cQLTpk2THoyImr56C+a9\n997Dnj178MQTTwAA3N3d+VIBImqQegtGrVbDwcFBa52hoaG0QETUfNT7TN4nn3wSKSkpUCgUqK2t\nxcqVK9GlSxddZCOiJq7eEUxkZCQiIiJw/vx52NjY4KeffkJkZKQushFRE1fvCMba2hpbtmzRRRYi\nambqLZi33377vq9JWrdunZRARNR81FswQ4YM0XxdWVmJnTt34sknn5Qaioiah3oL5tVXX9Vanjhx\nIoYOHSotEBE1Hw/9UoH8/Hz8+uuvMrIQUTNT7wimXbt2mjkYtVqN9u3bY8mSJdKDEVHT98CCEULg\nxIkT6NixIwDAwMDggW9CRUR0twdeIikUCgQEBMDQ0BCGhoYsFyJ6KPXOwTz33HNIS0vTRRYiambq\nvESqqamBkZERDh8+jKioKDg7O6N169YQQkChULB0iKhedRbMnZFLXFycLvMQUTNSZ8EIIQDc/jRH\nIqI/o86CKS4uRkRERJ03/OCDD6QEIqLmo86Cqa2tRXl5uWYkQ0T0sOosGFtbW3zyySe6zEJEzUyd\nD1Nz5EJEj6rOgklISNBlDiJqhuosmPbt2+syBxE1Q/zgNSKShgVDRNKwYIhIGhYMEUnDgiEiaVgw\nRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE\n0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcFIMnHiq/juu204ceJHnDjxI7Zv/yd8fPpr7ePo6IDI\nyAhkZCTj1KmfsGvXFjg7O+opMT2M2bNnIyUlBdeuXcOVK1ewa9cudO/evc79165dCyEEZsyYocOU\n+lfnZ1PTo7l0qQiff74cKtV5GBgYYMyYF7FmzTK89NI4ZGefhb19R/zrXzHYuXMPVq1ah+vXy+Ds\n7IiKigp9R6cGGDRoEFavXo3U1FQoFAosWLAA+/fvR7du3VBSUqK179ixY9GnTx8UFBToKa3+sGAk\n2b//oNby0qWrEBj4Cnr2dEd29lnMnBmCw4ePYvHipZp9LlxoeT+ATdWwYcO0lidOnIhr166hX79+\n2LNnj2b9U089hRUrVmDIkCHYu3evrmPqHS+RdMDAwACjRg2DmZkZ0tIyoFAo4Os7EGfPnkN09Gqk\npiYiLi4WI0f66Tsq/UkWFhYwNDTUGr0YGhpi8+bNWLhwIbKzs/WYTn84gpHIxaUTtm/fCBMTY1RU\nVODdd99HTk4uLC2fgLl5awQFTcGyZf+LL75Ygeeffw7Lli3GzZs3ceDAIX1Hp4e0YsUKpKen4+jR\no5p18+fPx2+//YY1a9boMZl+sWAkOndOhVGjXkGbNhYYNmwIwsM/xfjxU1Baeg0AsH9/Itav3wgA\nyMrKQY8e3TBhwqssmCZm6dKl8Pb2hre3N9RqNQBgwIABmDRpEjw8PPScTr+kXSLdunULc+fOhY+P\nD3r27Al/f38kJSXJOl2jVF1dg19/vYBffjmN8PAvkZWVg7femoCSkhJUV1fj7NlzWvvn5ubDzs5W\nT2npz4iIiMC4cePg6+uL/Px8zXofHx/Y2tri0qVLqK6uRnV1NZ5++ml8/vnnuHDhgh4T65a0EUxN\nTQ1sbW2xceNG2NnZISkpCdOnT8fu3bthb28v67SNmkJhAGNjY1RX1+DkyVNwcnpaa7ujowMKCwv1\nE44e2vLly/Haa69h0KBByMnJ0dq2evVqbN++XWvd999/j82bNyMqKkqXMfVKWsGYmZlh2rRpmmUf\nHx/Y29vj1KlTLaJgZs36HyQmHkJhYRHMzc0wevQIeHr2xuTJIQCAdetisHJlOFJT03D0aAo8Pftg\n1Cg/TJ36vp6TU0OsWrUKEydOhL+/P0pKSmBjYwMAKC8vx40bN1BcXIzi4mKt21RXV+Py5cs4c+aM\nPiLrhc7mYK5evQqVSoVOnTrp6pR6ZWn5BCIiFsPS0hJlZeXIyTmDN98MRnLyEQDADz8kIjR0AYKC\npuCTT2ZBpTqPmTP/jsTEZD0np4YIDg4GABw4cEBr/bx58zB//nx9RGqUFEIIIfsk1dXVePvtt/HU\nU09hwYIF92y/ePEiBg8ejOpqY/CR8+YpP/+kviOQJEZGRnByckJCQsI9VyfSf5vVajVmzZoFpVKJ\njz/+WPbpiKgRkXqJJIRAaGgorl69iqioKCiVSpmnI6JGRmrBhIWFIS8vD9HR0WjVqpXMUxFRIySt\nYAoKCrB161YYGxvD29tbs37+/PkYPXq0rNMSUSMirWA6dux4z3MDiKhl4UM2RCQNC4aIpGHBEJE0\nLBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOC\nISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgi\nkoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJp\nWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgiksZI\n3wEAoLa29v+/EgDU+oxCkhgZNYofNZLgzv/tf3+P79qm6zD3U1xcDABQKqv1nIRkcXJy0ncEkqy4\nuBgODg5a6xRCCKGnPBqVlZXIzMyElZUVDA0N9R2HiB5CbW0tiouL4erqilatWmltaxQFQ0TNEyd5\niUgaFgwRScOCISJpWDBEJA0LRsc2bdqEMWPGwNXVFbNnz9Z3HHqMbt26hblz58LHxwc9e/aEv78/\nkpKS9B1LrxrF82BaEmtrawQFBSE5ORlVVVX6jkOPUU1NDWxtbbFx40bY2dkhKSkJ06dPx+7du2Fv\nb6/veHrBgtGxF154AQDwyy+/oKioSM9p6HEyMzPDtGnTNMs+Pj6wt7fHqVOnWmzB8BKJSJKrV69C\npVKhU6dO+o6iNywYIgmqq6sxc+ZMBAQEwNnZWd9x9IYFQ/SYqdVqzJo1C0qlEh9//LG+4+gV52CI\nHiMhBEJDQ3H16lVERUVBqVTqO5JesWB0rKamBrW1tVCr1aitrUVVVRUMDQ35dgbNRFhYGPLy8hAd\nHX3PC/9aIr7YUcdWrlyJVatWaa0LCQnRevSBmqaCggL4+vrC2NhY6w/G/PnzMXr0aD0m0x8WDBFJ\nw0leIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LppkyNDSEh4cHXF1d8fLLL6OiouJPH+vgwYMY\nNWoUAGDXrl1YsmRJnfuWlpZi9erVD32OefPm4R//+EeD199t0qRJ2L59e4PPpVKp4Orq+tAZ6eGx\nYJopU1NTZGRkIDMzE8bGxlizZo3WdiEE1OqH/5C70aNHP/CNsv5swVDzxIJpAfr374/c3FyoVCo8\n88wzCAoKQq9evXDhwgXEx8fDy8sLvXr1wssvv4zy8nIAwL59+9C1a1d4e3tjx44dmmPFxMQgJCQE\nAFBUVISAgAC4u7vD3d0dR44cwezZs5GXlwcPDw98+OGHAIDw8HD06dMHbm5uCAsL0xxr0aJFcHFx\nwZAhQ5CTk1Pv9xEVFYU+ffrA3d0dY8eO1RqV7d+/H/3790eXLl2wZ88eALc/r+fDDz/UnHvt2rWP\nfmfSQ2HBNHM1NTXYu3cvevToAQDIycnB66+/jvT0dLRu3RoLFy7E/v37kZaWht69eyMiIgKVlZV4\n++23sXv3biQnJ+Py5cv3PfZ7772HgQMH4sSJE0hLS0P37t2xZMkSODs7IyMjA+Hh4YiPj8fZs2eR\nkpKCjIwMHD9+HIcOHcLx48exZcsWpKenY8eOHUhNTa33exkzZgxSU1Nx4sQJPPPMM1i/fr1mm0ql\nQlJSEr799ltMnToVlZWVWL9+Pdq2bYvU1FSkpqYiKioK+fn5j+eOpQbhK+yaqZs3b8LDwwPA7RHM\n5MmTUVhYCAcHB3h6egIAfvrpJ5w+fRr9+vUDcPs9Zb28vJCdnQ1HR0d07twZADBhwgSsW7funnMc\nOHAA//znPwHcnvNp27YtSkpKtPaJj49HfHw8evbsCQAoLy/H2bNnUVZWhoCAAJiZmQFAg16rk5mZ\nib///e8oLS1FeXk5/Pz8NNteeeUVGBgYoHPnznByckJ2djbi4+Nx8uRJzfzMtWvXcPbsWXTp0qXh\ndyQ9EhZMM3VnDuaPWrdurflaCIGhQ4di8+bNWvtkZGRAoVA8lhxCCMyZMwfvvPOO1vrly5c/9Dkm\nTZqEuLg4uLu7IyYmBgcPHtRs++OxFAoFhBBYuXKlVhEBt0c7pBu8RGrBPD098eOPPyI3NxcAUFFR\ngTNnzqBr167Iz89HXl4eANxTQHcMHjwYkZGRAG7Pd1y/fh0WFhYoKyvT7OPn54cNGzZo5nYKCgpw\n5coVDBgwADt37sTNmzdRVlaG3bt315u3rKwMtra2qK6uRmxsrNa2bdu2Qa1WIy8vD+fOnYOLiwv8\n/PwQGRmJ6upqAMCZM2dw48aNh7yX6FFwBNOCWVlZISYmBuPGjdN8wsHChQvRpUsXrFu3DiNHjoSl\npSW8vb2RmZl5z+1XrFiBv/3tb1i/fj0MDQ0RGRkJLy8v9OvXD66urhg+fDjCw8ORlZUFLy8vAIC5\nuTk2bdqEXr164dVXX4WHhwccHBzQv3//evN++umn6Nu3LxwcHNCjRw+tInNxccHAgQNRVFSENWvW\noFWrVpgyZQpUKhV69eoFIQSsrKwQFxf3mO49agi+XQMRScNLJCKShgVDRNKwYIhIGhYMEUnDgiEi\naVgwRCQNC4aIpGHBEJE0/wd9sMv+9+ckiQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1178d5650>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    0.5s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model Perceptron(alpha=0.0001, class_weight=None, eta0=1.0, fit_intercept=True,\n",
      "      n_iter=5, n_jobs=1, penalty=None, random_state=0, shuffle=True,\n",
      "      verbose=0, warm_start=False)\n",
      "Grid Search Best Score: 0.63625\n",
      "Grid Search Best Params: {'pca__n_components': 4}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.70      0.99      0.82       140\n",
      "          2       0.00      0.00      0.00        60\n",
      "\n",
      "avg / total       0.49      0.69      0.57       200\n",
      "\n",
      "Accuracy on Test Data: 0.695\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHgxJREFUeJzt3XtYVGXiB/DvAQaQiyQCimKsoGKKIGoJiavihdKVoDbb\n7KK75q4XNCvNC7VaqbldvGSJiQqtmrq2ypqGoah4IS8hqKioEKNcDCEDAeM67+8PH+fnhIgK7xwZ\nvp/n4Xmc85455zsDfH3POTODIoQQICKSwEztAERkulgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCa\nkXnz5qFNmzZQFAUxMTEN3p5Wq4WiKPjxxx8bHq4JGDhwIMLDw9WO0aQofB2MuvLz87Fw4ULs2LED\nOTk5cHJygo+PD6ZMmYLhw4c32n7S0tLQo0cPbN26FQEBAXBwcECLFi0atM2amhoUFBTAyckJFhYW\njZTUeLRaLTp27Ijjx4+jT58+9a5/7do1aDQa2NvbGyGdaWh6PxUmRKvVol+/frC3t8eHH34IX19f\n6HQ6JCQkYMKECbh8+XKj7SsjIwMAEBoaCkVRGmWb5ubmaNu2baNs62FWWVkJS0tLODo6qh2l6RGk\nmqefflq4urqKkpKSWmPXrl3T//vSpUsiNDRU2NnZCTs7OxEWFiays7P143PnzhXdu3cXGzduFB4e\nHsLOzk4888wzoqCgQD8OwOBLCCHGjBkjRowYYbDfW9u65dSpUyIoKEjY29sLOzs74ePjI/bu3SuE\nECIrK0sAEMePH9evn5iYKJ544glhZWUlXFxcxLRp00RFRYV+fMCAAWLixIli9uzZonXr1sLZ2Vm8\n9dZboqamps7nKTo6Wtja2orvvvtOeHl5iRYtWoiRI0eKoqIisWXLFtGpUyfRsmVL8fLLL4sbN27o\n7xcXFycCAwPFI488Ilq1aiWGDRsmzp49qx///XMyYMAAg+dl0aJFon379sLZ2VmfffLkyUIIIdLT\n04WNjY346quvDPan0WjEDz/8UOdjaW54DkYl165dw65duxAeHg47O7ta461atQIACCEQGhqK/Px8\n7N27F/v27UNeXh5CQ0Mhbju61Wq12Lx5M7Zt24b4+HikpKQgIiICADB9+nRERUUBAK5cuYIrV67c\nc87Ro0fD1dUVx44dQ0pKCubNmwdra+s7rpubm4unn34afn5+SElJwZo1a7Bx40bMnj3bYL0NGzbA\nwsICSUlJ+Pzzz7F06VJs3rz5rjkqKirw6aefYsOGDUhISMCPP/6IP//5z/jqq6/w3//+F7Gxsdix\nYwdWrFihv09ZWRmmTZuGY8eOYf/+/XBwcMDIkSNRWVkJADh27BgAYNeuXbhy5Qq2bt2qv29iYiJO\nnTqFXbt2ISEhoVYeLy8vLFmyBOHh4cjMzERBQQHGjh2LiIgI+Pv739uT2xyo3XDN1dGjRwUAsXXr\n1ruuFx8fL8zMzERWVpZ+WWZmplAURezevVsIcXPWYWVlJYqKivTrzJ8/X3h6eupvb9myRfz+230v\nMxh7e3sRExNzx2y/n8HMmTNHeHp6GsxGoqOjhaWlpSgrKxNC3JwF+Pv7G2xnyJAhYty4cXU+B9HR\n0QKASE9P1y976623hJmZmX6WVtfjuV1paakwMzMTBw8evGP+27fj5OQkysvLDZbfPoO5JTQ0VPTt\n21cMHz5cPPnkk6K6urrO/TdHnMGoRNzjufVz586hXbt2+MMf/qBf5uHhgXbt2uHs2bP6Ze7u7nBw\ncNDfbteuHa5evdrgnG+++SZee+01BAUFYcGCBUhPT79r1oCAAJiZ/f+PVWBgICorK/XngADAx8fH\n4H73ktXKygpeXl76223atEHbtm3h5ORksOz27WRmZmL06NHw9PREy5Yt0aZNG+h0uns6t+Xt7Q0r\nK6t611u9ejUyMjJw4MABrF+/Hubm5vXepzlhwaikc+fOUBQF586du+t6Qog6T8revlyj0dQa0+l0\nd922mZlZraKrqqoyuD1v3jycPXsWoaGhSEpKgo+PD9auXWv0rL+/SqUoSr3bGTlyJAoKCvDll1/i\n6NGjSElJgYWFhf4Q6W5sbW3rXQe4eXWuuLgY5eXlyM3Nvaf7NCcsGJU4OjoiODgYn3/+OUpLS2uN\nFxUVAQC6deuG3NxcaLVa/dhPP/2EvLw8dOvWrUEZnJ2da52PSU1NrbVe586dMXXqVOzcuRPjxo3D\n6tWr77i9bt264YcffjD4JT906BAsLS3h6enZoKz365dffsG5c+cwZ84cDBkyBI899hhKSkpQXV2t\nX8fS0hLAzcvtD6KoqAivvvoqpk+fjsmTJ+OVV17B9evXGyW/qWDBqGjFihUQQqBPnz7YsmULzp8/\nj/T0dERGRuoPI4YMGQJfX1+89NJLSE5Oxo8//oiXXnoJvXr1QlBQUIP2HxQUhJSUFKxduxYZGRn4\n6KOPcPjwYf34b7/9hsmTJ2P//v3QarU4evQoDh06VGexTZo0CXl5eZg0aRLOnTuHnTt3YtasWQgP\nD4eNjU2Dst6vVq1awcnJCVFRUcjIyEBiYiImTJhgMBNycXFBixYt8P333yM/Px/FxcX3tY8JEybA\nyckJ77//Pv71r3/B3t4ekydPbuyH0qSxYFTUsWNHnDhxAkOHDsXMmTPh4+ODoKAgbN++HV9++SWA\nm9P+2NhYODs7Y+DAgRg0aBDatm2L2NjYBr+eJTg4GHPnzkVERAR69+4NrVaLSZMm6cfNzc3x66+/\nYsyYMfDy8kJYWBgCAgKwePHiO26vffv2iIuLQ0pKCnr27Im//e1vePHFF7Fw4cIG5XwQZmZm2Lx5\nM06dOgVvb29MnjwZH3zwgcF5FQsLC3z22WdYvXo12rVrh2eeeeaet79u3Tps374dGzZsgEajgZWV\nFb7++mt888032Lhxo4yH1CTxlbxEJA1nMEQkDQuGiKRhwRCRNCwYIpLmoXg3dXl5OdLS0uDs7MxX\nQhI1Mbc+tsPb27vW+9QeioJJS0vDSy+9pHYMImqADRs21PpcnYeiYJydnQHcDNgcPl+kOerY0UPt\nCCSJhYUFHn20g/732GBMhTy13Dosatu2Ldzc3FROQzLc/hJ9Mk13Or3Bk7xEJA0LhoikYcEQkTQs\nGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4Ih\nImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCaSQHDhxASEgI2rdvD0VREBMTYzD+7rvv\nomvXrrC1tUWrVq0wePBgJCUlGayTmZmJsLAwODs7o2XLlhg1ahTy8/ON+CioIfr374///S8WOTnZ\nEEKHMWPGqB1JdSyYRlJaWgpvb28sW7YMLVq0qDXu5eWFL774AqdPn8ahQ4fQsWNHPPXUU/oCKSsr\nw7BhwyCEQEJCAg4fPozKykqMHDkSOp3O2A+HHoCdnR3S0s7g9den4caNG2rHeTiIh0B2drbo0qWL\nyM7OVjtKo7C1tRXR0dF3Xae4uFgAELt27RJCCPH9998LRVHEtWvX9OsUFRUJRVHE7t27ZcY1CkBp\nVl8lJSVizJixqucwxpeFhabO31/OYFRQWVmJVatWoWXLlujZsycAoKKiAoqiwNraWr+etbU1zMzM\ncOjQIbWiEjUIC8aIduzYATs7O1hbW2PJkiXYvXs32rRpAwDw9/eHnZ0dZsyYgbKyMpSVlWH69Omo\nqanBlStXVE5O9GBYMEY0aNAgpKamIikpCU899RRGjRqlLw9nZ2ds2bIFcXFxsLe3h4ODA4qKitCr\nVy+Ym5urnJzowUgtmPXr1+PZZ5+Ft7c3Zs2aJXNXTYKtrS06deoEf39/rFmzBhqNBqtXr9aPDxs2\nDJmZmbh69SoKCwuxbt065ObmomPHjiqmJnpwFjI37uLigkmTJuHgwYOoqKiQuasmSafT3fF5cXJy\nAgDs3bsXV69eRUhIiLGjETUKqQUzbNgwAMDp06dN/vUcpaWlyMjIAHCzOC5fvozU1FQ4OjrikUce\nwUcffYSRI0fC1dUVBQUF+OKLL5CTk4NRo0bptxEdHY2uXbvCxcUFP/zwA15//XW88cYb8PLyUuth\n0X24NUMFADMzMzz66KPw9fXFtWvXkJ2drXI6lRjjEuXixYvFzJkz6xw3hcvU+/btEwBqfY0ZM0aU\nlZWJ0NBQ4erqKiwtLYWrq6sICQkRR44cMdjGzJkzRZs2bYRGoxGdO3cWn376qdDpdCo9osal9qVU\nY3wNGDDwjo89Ojpa9WxqXaaWOoNpTgYOHAghRJ3j27Ztq3cbixYtwqJFixozFhlRYmIiFIXXTW7H\nZ4OIpGHBEJE0Ug+RqqurUVNTA51Oh5qaGlRUVMDc3BwWFjwyI2oOpM5gIiMj4ePjg1WrVmH79u3w\n8fFBZGSkzF0S0UNE6lRiypQpmDJlisxdENFDjOdgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQs\nGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4Ih\nImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQWdQ1c\nv379rnds2bJlo4chItNSZ8F0794diqJACKFfduu2oii4fPmyUQISUdNVZ8FkZ2cbMwcRmaB7Ogez\nadMmLFy4EACQk5OD5ORkqaGIyDTUWzDh4eHYt28f1q1bBwCwsbHBhAkTpAcjoqavzkOkW5KSknDi\nxAn4+fkBABwdHVFZWSk9GBE1ffXOYDQaDXQ6HRRFAQD88ssvMDPj1W0iql+9TTF58mQ899xzKCgo\nwNy5cxEYGIiZM2caIxsRNXH1HiK9+uqr6N27N/bs2QMA2LJlC7y9vaUHI6Kmr96CAYCamhpoNBoo\nigKdTic7ExGZiHoPkRYsWIAXX3wReXl5yMnJwejRo/Hhhx8aIxsRNXH1zmDWr1+P5ORk2NjYAAAi\nIiLQu3dvzJ49W3o4Imra6p3BuLu7o7q6Wn+7uroaHh4eUkMRkWmocwbzxhtvQFEU2NjYoHv37ggO\nDoaiKIiPj0dgYKAxMxJRE1Vnwdy6UtS9e3eMGDFCv9zf319+KiIyCXUWzLhx44yZg4hMUL0neTMz\nMxEREYGzZ8+ivLxcv/zChQtSgxFR01fvSd6xY8fir3/9K4QQiIuLw6hRo/CXv/zFGNmIqImrt2Bu\n3LiB4OBgAICnpyfmz5+Pffv2SQ9GRE1fvYdIVlZWEELA09MTK1euRPv27XH16lVjZCOiJq7eglmy\nZAlKS0vx2WefISIiAsXFxVi7dq0xshFRE1dvwfTt2xcAYG9vr//QKSKie1FnwYSFhek/A+ZOtm7d\nKiUQEZmOOgsmPDzcmDkAADPmLIO1tZ3R90vGIOpfhZqour+3dRbM4MGDpUQhouaDn31JRNKwYIhI\nmnsumIqKCpk5iMgE1Vswx44dQ48ePdC5c2cAwMmTJzFlyhTpwYio6au3YKZOnYodO3agdevWAABf\nX1++VYCI7km9BaPT6eDu7m6wzNzcXFogIjId9b6St0OHDjh27BgURUFNTQ2WL1+OLl26GCMbETVx\n9c5gIiMjsXjxYly+fBlt2rTBkSNHEBkZaYxsRNTE1TuDcXFxwaZNm4yRhYhMTL0FM378+Du+J2nV\nqlVSAhGR6ai3YIYMGaL/d3l5ObZt24YOHTpIDUVEpqHegnnhhRcMbr/yyisYOnSotEBEZDru+60C\nWVlZuHTpkowsRGRi6p3BtGrVSn8ORqfTwdHREYsWLZIejIiavrsWjBACJ0+eRPv27QEAZmZmd/0Q\nKiKi2931EElRFISFhcHc3Bzm5uYsFyK6L/Weg3niiSdw4sQJY2QhIhNT5yFSdXU1LCwscOjQIURF\nRcHT0xO2trYQQkBRFJYOEdWrzoK5NXOJjY01Zh4iMiF1FowQNz/I19PT02hhiMi01FkwBQUFWLx4\ncZ13fPPNN6UEIiLTUWfB1NTUoLS0VD+TISK6X3UWjKurK/75z38aMwsRmZg6L1Nz5kJEDVVnwSQk\nJBgzBxGZoDoLxtHR0Zg5iMgE8Q+vEZE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRh\nwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYM\nEUnDgiEiaVgwRCRNnX86lhrOwcEOz4cNhk+PzrC2tsLVgl+x7uudOH/hkn6dZ0YOwID+vWFrY42f\nsnKx7uvvkHelQMXU1BATJ07EjBkz4OrqijNnzmDatGk4dOiQ2rFUwxmMJC1aWGHO238DFAVLln+N\nOf/8Ahs2xeH69TL9OsOD+yF4aAA2bIrD+wujcL2kDNPfeAXWVpYqJqcHNWrUKCxbtgwLFy6En58f\nkpKSEBcXhw4dOqgdTTUsGEmGB/dDcXEJVkfHIkubh8JfinAuPQtXfi7UrzN0SF98t+swkk+cQ25e\nAVZHx8La2hL+fXuomJwe1JtvvomYmBisXr0a6enpmDp1Kq5cuYKJEyeqHU01PESSxK9nV6SdycDE\n8c+hq1dHFBWX4MChE0jYdxwA4Oz0CB5xsMeZM5n6+1RVVePCxUvo5OGG/QeS1YpOD0Cj0aB37974\n5JNPDJbHx8fjySefVCmV+jiDkcTFuRWCBj6OgsIifLpsPXYnHMWfw4Zg8KDHAQAOLe0AAMUlpQb3\nu369DA4OdkbPSw3j5OQECwsL5OfnGyzPz89H27ZtVUqlPs5gJFEUBdpLefhmWwIA4HL2z2jj4oig\ngY/rZzEAAFH7vuIOy6hpEL/75imKUmtZcyJtBlNZWYk5c+Zg0KBB8PPzQ2hoKBITE2Xt7qFTVFyC\nvDzDq0F5PxeitaMDAKD4+s2Zy62ZzC0tW9ri+nXDWQ09/AoLC1FdXV1rtuLi4lJrVtOcSCuY6upq\nuLq6Yt26dUhOTsbrr7+OadOmIScnR9YuHyoZGdlo27a1wbK2Lq1R+EsxAKCgsAhFxSXo1s1DP25h\nYY4undyR8VPzeI5MSVVVFZKTkzF06FCD5UOHDkVSUpJKqdQnrWBsbGwwZcoUuLm5wczMDIMGDYKb\nmxvOnDkja5cPlfg9R+Dh4YY/De8PF+dW6NO7G4YMfgJ79///4dHuPUcx4qlA9PbrivbtnPHaX0NR\nXlGJI0dPq5icHtTixYsxduxYjBs3Dl27dsXSpUvRrl07rFy5Uu1oqjHaOZjCwkJotVp06tTJWLtU\nVdalPCxfsRnPhQYhZMQf8cu1Ymz73z6Dgvnu+8PQWFrg5dHDYWvTAplZOfh06TqUV1SqmJwe1H/+\n8x+0bt0a77zzDlxdXZGWlobhw4fj8uXLakdTjSKMcAaqqqoK48ePx6OPPor333+/1nhOTg4GDx6M\nXn1DYG3NKyimKCZqntoRSBILCwt4eHggISEBbm5uBmPSL1PrdDq8/fbb0Gg0ePfdd2XvjogeIlIP\nkYQQiIiIQGFhIaKioqDRaGTujogeMlILZu7cucjMzER0dDSsra1l7oqIHkLSCiY3NxebN2+GpaUl\nAgMD9cvfe+89hISEyNotET1EpBVM+/btcf78eVmbJ6ImgO9FIiJpWDBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVD\nRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQk\nDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQWagcAgJqaGgBA\nRcUNlZOQLBYWD8WPGklw63t76/fYYMzYYe6koKAAAHAmdY/KSUgWDw8PtSOQZAUFBXB3dzdYpggh\nhEp59MrLy5GWlgZnZ2eYm5urHYeI7kNNTQ0KCgrg7e0Na2trg7GHomCIyDTxJC8RScOCISJpWDBE\nJA0LhoikYcEY2fr16/Hss8/C29sbs2bNUjsONaLKykrMmTMHgwYNgp+fH0JDQ5GYmKh2LFU9FK+D\naU5cXFwwadIkHDx4EBUVFWrHoUZUXV0NV1dXrFu3Du3atUNiYiKmTZuGb7/9Fm5ubmrHUwULxsiG\nDRsGADh9+jTy8/NVTkONycbGBlOmTNHfHjRoENzc3HDmzJlmWzA8RCKSpLCwEFqtFp06dVI7impY\nMEQSVFVVYfr06QgLC4Onp6facVTDgiFqZDqdDm+//TY0Gg3effddteOoiudgiBqREAIREREoLCxE\nVFQUNBqN2pFUxYIxsurqatTU1ECn06GmpgYVFRUwNzfnxxmYiLlz5yIzMxPR0dG13vjXHPHNjka2\nfPlyfP755wbLwsPDDa4+UNOUm5uLoKAgWFpaGvyH8d577yEkJETFZOphwRCRNDzJS0TSsGCISBoW\nDBFJw4IhImlYMEQkDQuGiKRhwZgoc3Nz9OzZE97e3nj++edx48aD/82p/fv3409/+hMAYPv27Vi0\naFGd6xYVFWHFihX3vY958+bhk08+uefltxs7diy++eabe96XVquFt7f3fWek+8eCMVEtWrRAamoq\n0tLSYGlpiZUrVxqMCyGg0+nue7shISF3/aCsBy0YMk0smGagf//+yMjIgFarxWOPPYZJkyahV69e\nyM7ORnx8PAICAtCrVy88//zzKC0tBQDs2rULXbt2RWBgILZu3arfVkxMDMLDwwEA+fn5CAsLg6+v\nL3x9fZGUlIRZs2YhMzMTPXv2xIwZMwAAH3/8MR5//HH4+Phg7ty5+m0tWLAAXl5eGDJkCM6fP1/v\n44iKisLjjz8OX19fPPfccwazsj179qB///7o0qULduzYAeDm3+uZMWOGft9ffvllw59Mui8sGBNX\nXV2NuLg49OjRAwBw/vx5vPrqq0hJSYGtrS3mz5+PPXv24MSJE+jTpw8WL16M8vJyjB8/Ht9++y0O\nHjyIn3/++Y7bnjp1KgYMGICTJ0/ixIkT6N69OxYtWgRPT0+kpqbi448/Rnx8PC5evIhjx44hNTUV\nycnJOHDgAJKTk7Fp0yakpKRg69atOH78eL2P5dlnn8Xx48dx8uRJPPbYY1izZo1+TKvVIjExETt3\n7sSECRNQXl6ONWvWwMHBAcePH8fx48cRFRWFrKysxnli6Z7wHXYm6rfffkPPnj0B3JzBjBs3Dnl5\neXB3d4e/vz8A4MiRIzh79iz69esH4OZnygYEBCA9PR0dO3ZE586dAQAvv/wyVq1aVWsfe/fuxb//\n/W8AN8/5ODg44NdffzVYJz4+HvHx8fDz8wMAlJaW4uLFiygpKUFYWBhsbGwA4J7eq5OWloZ33nkH\nRUVFKC0tRXBwsH5s1KhRMDMzQ+fOneHh4YH09HTEx8fj1KlT+vMzxcXFuHjxIrp06XLvTyQ1CAvG\nRN06B/N7tra2+n8LITB06FBs3LjRYJ3U1FQoitIoOYQQmD17Nv7xj38YLF+6dOl972Ps2LGIjY2F\nr68vYmJisH//fv3Y77elKAqEEFi+fLlBEQE3ZztkHDxEasb8/f1x+PBhZGRkAABu3LiBCxcuoGvX\nrsjKykJmZiYA1CqgWwYPHozIyEgAN893XL9+Hfb29igpKdGvExwcjLVr1+rP7eTm5uLq1av44x//\niG3btuG3335DSUkJvv3223rzlpSUwNXVFVVVVdiwYYPB2JYtW6DT6ZCZmYmffvoJXl5eCA4ORmRk\nJKqqqgAAFy5cQFlZ2X0+S9QQnME0Y87OzoiJicGLL76o/wsH8+fPR5cuXbBq1SqMGDECTk5OCAwM\nRFpaWq37L1u2DH//+9+xZs0amJubIzIyEgEBAejXrx+8vb3x9NNP4+OPP8a5c+cQEBAAALCzs8P6\n9evRq1cvvPDCC+jZsyfc3d3Rv3//evN+8MEH6Nu3L9zd3dGjRw+DIvPy8sKAAQOQn5+PlStXwtra\nGq+99hq0Wi169eoFIQScnZ0RGxvbSM8e3Qt+XAMRScNDJCKShgVDRNKwYIhIGhYMEUnDgiEiaVgw\nRCQNC4aIpGHBEJE0/wegcokgjYGEfQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1185f02d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    0.4s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model PassiveAggressiveClassifier(C=1.0, class_weight=None, fit_intercept=True,\n",
      "              loss='hinge', n_iter=5, n_jobs=1, random_state=None,\n",
      "              shuffle=True, verbose=0, warm_start=False)\n",
      "Grid Search Best Score: 0.6625\n",
      "Grid Search Best Params: {'pca__n_components': 10}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.73      0.69      0.71       140\n",
      "          2       0.36      0.42      0.39        60\n",
      "\n",
      "avg / total       0.62      0.60      0.61       200\n",
      "\n",
      "Accuracy on Test Data: 0.605\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHvJJREFUeJzt3X1czXfjP/DX6XRDN24LKVdTkptUMpsmF1o0c1d2uV9m\nY5spZmaGrk02o21mNi5tci/Dw6QfdiFEmJuS3LQktY6RVrkphUrnvL9/+DmXs6SM9zl1vJ6Ph8ej\n83l/zufz6tDL+/M+NymEEAJERBKYGDoAERkvFgwRScOCISJpWDBEJA0LhoikYcEQkTQsmGdIeHg4\nmjdvDoVCgdWrVz/x8VQqFRQKBU6cOPHk4eqAXr16ITQ01NAx6hQFXwdjWHl5eZg3bx527NiBy5cv\nw9bWFh4eHpg0aRJeffXVp3ae1NRUdOrUCTExMfDx8UHDhg1Rv379JzqmWq1GQUEBbG1tYWpq+pSS\n6o9KpULr1q2RlJSE559/vtr9r1+/DjMzM9jY2OghnXGoe/8qjIhKpUL37t1hY2OD+fPnw9PTExqN\nBvv27cOECRPwxx9/PLVzZWZmAgACAwOhUCieyjGVSiVatGjxVI5Vm5WXl8Pc3BxNmjQxdJS6R5DB\n9OvXT9jb24vi4uJKY9evX9d+ffHiRREYGCisra2FtbW1CAoKEpcuXdKOz549W3Ts2FFs2LBBODs7\nC2trazF48GBRUFCgHQeg80cIId544w3Rv39/nfPeP9Z9Z86cEX5+fsLGxkZYW1sLDw8PER8fL4QQ\nIjs7WwAQSUlJ2v0TEhLECy+8ICwsLESzZs3ElClTRFlZmXa8Z8+e4r333hMzZ84UTZs2FXZ2duLD\nDz8UarW6ysdp1apVwsrKSvz3v/8Vbm5uon79+mLgwIGisLBQbN68WbRp00Y0aNBAvP766+L27dva\n++3cuVP4+vqKRo0aicaNG4u+ffuKtLQ07fhfH5OePXvqPC4RERHCwcFB2NnZabOHhIQIIYRIT08X\nlpaWYs2aNTrnMzMzE0ePHq3ye3nWcA3GQK5fv45du3YhNDQU1tbWlcYbN24MABBCIDAwEHl5eYiP\nj8f+/ftx5coVBAYGQjxwdatSqbBp0yZs3boVcXFxSElJQVhYGABg2rRpiIqKAgDk5uYiNze3xjlH\njRoFe3t7JCYmIiUlBeHh4ahXr95D983JyUG/fv3QuXNnpKSkYMWKFdiwYQNmzpyps9/69ethamqK\nI0eOYMmSJVi0aBE2bdr0yBxlZWX45ptvsH79euzbtw8nTpzAv/71L6xZswZbtmxBbGwsduzYgaVL\nl2rvc+vWLUyZMgWJiYk4cOAAGjZsiIEDB6K8vBwAkJiYCADYtWsXcnNzERMTo71vQkICzpw5g127\ndmHfvn2V8ri5ueHbb79FaGgosrKyUFBQgLFjxyIsLAzdunWr2YP7LDB0wz2rjh8/LgCImJiYR+4X\nFxcnTExMRHZ2tnZbVlaWUCgUYs+ePUKIe7MOCwsLUVhYqN1n7ty5wsXFRXt78+bN4q9/3TWZwdjY\n2IjVq1c/NNtfZzCzZs0SLi4uOrORVatWCXNzc3Hr1i0hxL1ZQLdu3XSO4+/vL8aNG1flY7Bq1SoB\nQKSnp2u3ffjhh8LExEQ7S6vq+3lQSUmJMDExEYcOHXpo/gePY2trK0pLS3W2PziDuS8wMFC8+OKL\n4tVXXxUvvfSSqKioqPL8zyLOYAxE1HBt/dy5c2jZsiWee+457TZnZ2e0bNkSaWlp2m1OTk5o2LCh\n9nbLli2Rn5//xDmnTp2K8ePHw8/PD1988QXS09MfmdXHxwcmJv/7Z+Xr64vy8nLtGhAAeHh46Nyv\nJlktLCzg5uamvd28eXO0aNECtra2OtsePE5WVhZGjRoFFxcXNGjQAM2bN4dGo6nR2pa7uzssLCyq\n3W/58uXIzMzEwYMHER0dDaVSWe19niUsGANxdXWFQqHAuXPnHrmfEKLKRdkHt5uZmVUa02g0jzy2\niYlJpaK7e/euzu3w8HCkpaUhMDAQR44cgYeHB1auXKn3rH99lkqhUFR7nIEDB6KgoAA//vgjjh8/\njpSUFJiammovkR7Fysqq2n2Ae8/OFRUVobS0FDk5OTW6z7OEBWMgTZo0QUBAAJYsWYKSkpJK44WF\nhQCADh06ICcnByqVSjv2+++/48qVK+jQocMTZbCzs6u0HnPq1KlK+7m6umLy5Mn45ZdfMG7cOCxf\nvvyhx+vQoQOOHj2q80N++PBhmJubw8XF5YmyPq5r167h3LlzmDVrFvz9/dG+fXsUFxejoqJCu4+5\nuTmAe0+3/x2FhYUYM2YMpk2bhpCQEAQHB+PmzZtPJb+xYMEY0NKlSyGEwPPPP4/Nmzfj/PnzSE9P\nR2RkpPYywt/fH56enhg9ejSSk5Nx4sQJjB49Gt7e3vDz83ui8/v5+SElJQUrV65EZmYmvvrqK/z6\n66/a8Tt37iAkJAQHDhyASqXC8ePHcfjw4SqLbeLEibhy5QomTpyIc+fO4ZdffsGMGTMQGhoKS0vL\nJ8r6uBo3bgxbW1tERUUhMzMTCQkJmDBhgs5MqFmzZqhfvz52796NvLw8FBUVPdY5JkyYAFtbW3z2\n2Wf48ssvYWNjg5CQkKf9rdRpLBgDat26NU6ePIk+ffrg448/hoeHB/z8/LBt2zb8+OOPAO5N+2Nj\nY2FnZ4devXqhd+/eaNGiBWJjY5/49SwBAQGYPXs2wsLC0KVLF6hUKkycOFE7rlQqcePGDbzxxhtw\nc3NDUFAQfHx8sHDhwocez8HBATt37kRKSgq8vLzw1ltvYeTIkZg3b94T5fw7TExMsGnTJpw5cwbu\n7u4ICQnB559/rrOuYmpqiu+//x7Lly9Hy5YtMXjw4Boff926ddi2bRvWr18PMzMzWFhY4KeffsLP\nP/+MDRs2yPiW6iS+kpeIpOEMhoikYcEQkTQsGCKShgVDRNLUindTl5aWIjU1FXZ2dnwlJFEdc/9j\nO9zd3Su9T61WFExqaipGjx5t6BhE9ATWr19f6XN1akXB2NnZAbgX8Fn4fJFn0dCh7xk6AkmiVt9F\nUVG29uf4QbWiYO5fFrVo0QKOjo4GTkMyKJXmho5Akj1seYOLvEQkDQuGiKRhwRCRNCwYIpKGBUNE\n0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQN\nC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBg\niEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aI\npGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4KRqLi4GFOmTIGTkxPq16+Pl156CUlJSTr7ZGRkYMiQ\nIWjUqBEsLS3h7e2Nc+fOGSgx/V3BwUNw+PAWfPDB+IeOT58+AYcPb8HIkYP0nMywTA0dwJiNHz8e\nZ86cwZo1a+Do6Ijo6Gj4+/sjLS0NDg4OyM7ORvfu3TFmzBjEx8ejUaNGSE9Ph7W1taGj02Po2NEV\nAwf6IzNT9dDxXr26oV07FxQUXNNvsFqAMxhJ7ty5gy1btiAiIgK9evVCmzZtEB4ejjZt2iAyMhIA\nEBYWhr59++Kbb76Bt7c3nJ2d8eqrr6JVq1YGTk81ZWVliU8/nYKIiKUoLi6pNN68uR3ef/8tzJmz\nCBUVagMkNCwWjCQVFRVQq9WoV6+ezvb69evj8OHD0Gg02L59Ozp06IBXXnkFdnZ26Nq1KzZt2mSg\nxPR3TJ8+AQcOHMXJk6mVxpRKE4SHT8GaNVtw8WKOAdIZHgtGEhsbG/j4+GDu3LnIycmBWq1GdHQ0\njh49itzcXOTn56OkpATz5s1D3759sWfPHowcORKjR4/Gjh07DB2famDgQH84OrZAVNTGh46PGzcC\nN2+WIDZ2t56T1R5cg5Fo3bp1eOutt+Do6AilUglvb2+MHDkSJ0+ehEajAQAMHjwYU6dOBQB4eXnh\nxIkT+M9//oMBAwYYMjpVo1WrlnjnnVEICfkEFRUVlca9vDqgX79eePPNaQZIV3tILZjo6GjExMQg\nIyMDAwYMQEREhMzT1TouLi5ISEjArVu3cPPmTdjb22P48OFo3bo1bG1tYWpqig4dOujcp3379ti4\n8eH/I1Lt4e7uhsaNG2Lt2m+120xNlfD07IDBg/tiw4b/h6ZNGyM2drnO+IQJr2Po0AEYMuQdQ8TW\nO6kF06xZM0ycOBGHDh1CWVmZzFPValZWVrCyssKNGzewe/dufPXVVzA3N0fXrl1x/vx5nX0zMjLg\n5ORkoKRUU4cOHUdwcKbOtlmzQnH5ci7Wrt2CwsKbiIs7qDO+cOEn2Lv3MLZt26vPqAYltWD69u0L\nADh79izy8vJknqpW2r17NzQaDdq1a4fMzEx89NFHcHNzw5tvvgkAmD59OoYNG4YePXrAz88P+/fv\nx8aNGxEbG2vg5FSdkpLbKCm5rbOttLQUN2+WIDv7EgDgxo0infGKCjWuXy/EpUtX9JbT0LjIK1FR\nURFCQ0PRrl07jBkzBr6+voiLi4OZmRkAIDAwEMuWLcOCBQvQqVMnLF68GGvXrkX//v0NnJzo6eAi\nr0TDhg3DsGHDHrnP2LFjMXbsWP0EIqkmTZr9yPGhQ9/TU5LagzMYIpKGBUNE0ki9RLr/alaNRgO1\nWo2ysjIolUqYmvLKjOhZIHUGExkZCQ8PDyxbtgzbtm2Dh4eH9n04RGT8pE4lJk2ahEmTJsk8BRHV\nYlyDISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0\nLBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOC\nISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKYVjVw8+bNR96xQYMGTz0MERmXKgumY8eOUCgUEEJo\nt92/rVAo8Mcff+glIBHVXVUWzKVLl/SZg4iMUI3WYDZu3Ih58+YBAC5fvozk5GSpoYjIOFRbMKGh\nodi/fz/WrVsHALC0tMSECROkByOiuq/KS6T7jhw5gpMnT6Jz584AgCZNmqC8vFx6MCKq+6qdwZiZ\nmUGj0UChUAAArl27BhMTPrtNRNWrtilCQkLw2muvoaCgALNnz4avry8+/vhjfWQjojqu2kukMWPG\noEuXLti7dy8AYPPmzXB3d5cejIjqvmoLBgDUajXMzMygUCig0WhkZyIiI1HtJdIXX3yBkSNH4sqV\nK7h8+TJGjRqF+fPn6yMbEdVx1c5goqOjkZycDEtLSwBAWFgYunTpgpkzZ0oPR0R1W7UzGCcnJ1RU\nVGhvV1RUwNnZWWooIjIOVc5gPvjgAygUClhaWqJjx44ICAiAQqFAXFwcfH199ZmRiOqoKgvm/jNF\nHTt2RP/+/bXbu3XrJj8VERmFKgtm3Lhx+sxBREao2kXerKwshIWFIS0tDaWlpdrtGRkZUoMRUd1X\n7SLv2LFj8eabb0IIgZ07d2LYsGEYMWKEPrIRUR1XbcHcvn0bAQEBAAAXFxfMnTsX+/fvlx6MiOq+\nai+RLCwsIISAi4sLfvjhBzg4OCA/P18f2Yiojqu2YL799luUlJTg+++/R1hYGIqKirBy5Up9ZCOi\nOq7agnnxxRcBADY2NtoPnSIiqokqCyYoKEj7GTAPExMTIyUQERmPKgsmNDRUnzkAAL6+AQCqLjWq\nuy5e/M3QEUgSU1PTKt8+VGXBvPzyy9ICEdGzgZ99SUTSsGCISJoaF0xZWZnMHERkhKotmMTERHTq\n1Amurq4AgNOnT2PSpEnSgxFR3VdtwUyePBk7duxA06ZNAQCenp58qwAR1Ui1BaPRaODk5KSzTalU\nSgtERMaj2lfytmrVComJiVAoFFCr1Vi8eDHatm2rj2xEVMdVO4OJjIzEwoUL8ccff6B58+Y4duwY\nIiMj9ZGNiOq4amcwzZo1w8aNG/WRhYiMTLUF8/bbbz/0PUnLli2TEoiIjEe1BePv76/9urS0FFu3\nbkWrVq2khiIi41BtwQwfPlzndnBwMPr06SMtEBEZj8d+q0B2djYuXrwoIwsRGZlqZzCNGzfWrsFo\nNBo0adIEERER0oMRUd33yIIRQuD06dNwcHAAAJiYmDzyQ6iIiB70yEskhUKBoKAgKJVKKJVKlgsR\nPZZq12BeeOEFnDx5Uh9ZiMjIVHmJVFFRAVNTUxw+fBhRUVFwcXGBlZUVhBBQKBQsHSKqVpUFc3/m\nEhsbq888RGREqiwYIQSAe7/NkYjo76iyYAoKCrBw4cIq7zh16lQpgYjIeFRZMGq1GiUlJdqZDBHR\n46qyYOzt7fHpp5/qMwsRGZkqn6bmzIWInlSVBbNv3z595iAiI1RlwTRp0kSfOYjICPEXrxGRNCwY\nIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEi\naVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQtGkuDgEdi5MwZnzx7D\n2bPHEBMTjd69/6kdX7BgLlSqVJ0/W7euN2BiehwzZsxAYmIiioqKkJ+fj23btqFjx446+6xatQpC\nCJ0/R48eNVBiw6jyd1PTk/nzzzx8+eVCZGdfhImJCV57bTCWLfsOAwcOR3p6BgDg0KGjmDp1hvY+\n5eV3DRWXHlOvXr2wdOlSJCUlQaFQ4LPPPsPevXvRoUMH3LhxQ7vfnj17EBwcrL1dXl5uiLgGw4KR\nZM+e/Tq3Fyz4Hq+/Phze3p7agikvL0dBwTVDxKMn9Morr+jcDg4ORlFREbp3744dO3Zot5eVlSEv\nL0/f8WoNXiLpgYmJCQYO7AdLS0skJ5/Sbu/atTNOnEhAfPwOzJ8fjqZN+et66yobGxsolUqd2QsA\n+Pr6Ii8vD+fPn8eyZctgZ2dnoISGwRmMRG5uroiJWQ8LC3Pcvn0b7777Ps6fvwAASEj4Fbt27cWl\nSzlwdGyJadMm46efVmDgwGG8VKqDvvvuO6SkpOissezatQsxMTHIzs7Gc889h7lz5yI+Ph5dunR5\nZi6VFEIIYegQly9fxssvv4yyMlMACkPHeWrMzEzRsqU9GjRogH79+mDEiNcwYsSbyMjIrLRvs2Z2\n+PXXOISGfoTdu/caIK1cFy/+ZugI0nzzzTcYMWIEfH19kZ2dXeV+9vb2uHjxIoYPH46tW7fqMaFc\npqamcHZ2xr59++Do6KgzJu0Sqby8HLNmzULv3r3RuXNnBAYGIiEhQdbpaqW7dytw8eIlnD37G776\nahHS0tIxbtyYh+6bn1+AP//MQ+vW/9BzSnoSCxcuxMiRI+Hn5/fIcgGA3NxcXL58Ga6urnpKZ3jS\nCqaiogL29vZYt24dkpOT8f7772PKlCm4fPmyrFPWeiYmJjA3N3/oWOPGjdC8eXPk51/Vcyr6uxYt\nWoRRo0bBz88P58+fr3b/pk2bwsHBAbm5uXpIVztIW4OxtLTEpEmTtLd79+4NR0dH/Pbbb5WmUcbo\n44+nID7+IHJz/4SVlRUGD+6Pbt264q23JsLSsj6mTAnBrl17kJ9fAEdHB0yf/j6uXbtmlJdHxmjJ\nkiUIDg5GYGAgbty4gebNmwMASkpKcOvWLVhZWSE8PBxbtmxBbm4unnvuOcyfPx/5+flGdXlUHb0t\n8l69ehUqlQpt2rTR1ykNys7OFt9+GwE7O1sUFxcjPT0DY8dOwMGDR2BhYYF27VwxZMhANGjQAPn5\nBTh2LBEhIdNw69ZtQ0enGggJCQEAxMfH62wPDw/HnDlzoFar0alTJ4wZMwaNGjVCbm4u9u/fj2HD\nhqGkpMQQkQ1CL4u8d+/exdtvv41//OMf+OyzzyqNG+siL/2PMS/yPusMssh7n0ajwfTp02FmZoZP\nPvlE9umIqBaReokkhEBYWBiuXr2KqKgomJmZyTwdEdUyUgtm9uzZyMrKwqpVq1CvXj2ZpyKiWkha\nweTk5GDTpk0wNzeHr6+vdvucOXMwaNAgWaclolpEWsE4ODjU6LUBRGS8+GZHIpKGBUNE0rBgiEga\nFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHB\nEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwR\nScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0\nLBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRSWNq\n6AAAoFar//9XwqA5SB5T01rxT40kuP93+7+f4wfG9B3mYQoKCgAAFhaVA5JxcHZ2NnQEkqygoABO\nTk462xRCCINPG0pLS5Gamgo7OzsolUpDxyGix6BWq1FQUAB3d3fUq1dPZ6xWFAwRGScu8hKRNCwY\nIpKGBUNE0rBgiEgaFoyeRUdHY8iQIXB3d8eMGTMMHYeeovLycsyaNQu9e/dG586dERgYiISEBEPH\nMqha8TqYZ0mzZs0wceJEHDp0CGVlZYaOQ09RRUUF7O3tsW7dOrRs2RIJCQmYMmUKtm/fDkdHR0PH\nMwgWjJ717dsXAHD27Fnk5eUZOA09TZaWlpg0aZL2du/eveHo6IjffvvtmS0YXiIRSXL16lWoVCq0\nadPG0FEMhgVDJMHdu3cxbdo0BAUFwcXFxdBxDIYFQ/SUaTQaTJ8+HWZmZvjkk08MHceguAZD9BQJ\nIRAWFoarV68iKioKZmZmho5kUCwYPauoqIBarYZGo4FarUZZWRmUSiU/zsBIzJ49G1lZWVi1alWl\nN/49i/hmRz1bvHgxlixZorMtNDRU59kHqptycnLg5+cHc3Nznf8w5syZg0GDBhkwmeGwYIhIGi7y\nEpE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEZKqVTCy8sL7u7uGDp0KG7fvv23j3XgwAEMGDAA\nALBt2zZERERUuW9hYSGWLl362OcIDw/HggULarz9QWPHjsXPP/9c43OpVCq4u7s/dkZ6fCwYI1W/\nfn2cOnUKqampMDc3xw8//KAzLoSARqN57OMOGjTokR+U9XcLhowTC+YZ0KNHD2RmZkKlUqF9+/aY\nOHEivL29cenSJcTFxcHHxwfe3t4YOnQoSkpKAAC7du1Cu3bt4Ovri5iYGO2xVq9ejdDQUABAXl4e\ngoKC4OnpCU9PTxw5cgQzZsxAVlYWvLy88NFHHwEAvv76a3Tt2hUeHh6YPXu29lhffPEF3Nzc4O/v\nj/Pnz1f7fURFRaFr167w9PTEa6+9pjMr27t3L3r06IG2bdtix44dAO79vp6PPvpIe+4ff/zxyR9M\neiwsGCNXUVGBnTt3olOnTgCA8+fPY8yYMUhJSYGVlRXmzp2LvXv34uTJk3j++eexcOFClJaW4u23\n38b27dtx6NAh/Pnnnw899uTJk9GzZ0+cPn0aJ0+eRMeOHREREQEXFxecOnUKX3/9NeLi4nDhwgUk\nJibi1KlTSE5OxsGDB5GcnIyNGzciJSUFMTExSEpKqvZ7GTJkCJKSknD69Gm0b98eK1as0I6pVCok\nJCTgl19+wYQJE1BaWooVK1agYcOGSEpKQlJSEqKiopCdnf10HliqEb7DzkjduXMHXl5eAO7NYMaN\nG4crV67AyckJ3bp1AwAcO3YMaWlp6N69O4B7nynr4+OD9PR0tG7dGq6urgCA119/HcuWLat0jvj4\neKxduxbAvTWfhg0b4saNGzr7xMXFIS4uDp07dwYAlJSU4MKFCyguLkZQUBAsLS0BoEbv1UlNTcW/\n//1vFBYWoqSkBAEBAdqxYcOGwcTEBK6urnB2dkZ6ejri4uJw5swZ7fpMUVERLly4gLZt29b8gaQn\nwoIxUvfXYP7KyspK+7UQAn369MGGDRt09jl16hQUCsVTySGEwMyZM/Huu+/qbF+0aNFjn2Ps2LGI\njY2Fp6cnVq9ejQMHDmjH/noshUIBIQQWL16sU0TAvdkO6QcvkZ5h3bp1w6+//orMzEwAwO3bt5GR\nkYF27dohOzsbWVlZAFCpgO57+eWXERkZCeDeesfNmzdhY2OD4uJi7T4BAQFYuXKldm0nJycH+fn5\n+Oc//4mtW7fizp07KC4uxvbt26vNW1xcDHt7e9y9exfr16/XGdu8eTM0Gg2ysrLw+++/w83NDQEB\nAYiMjMTdu3cBABkZGbh169ZjPkr0JDiDeYbZ2dlh9erVGDlypPY3HMydOxdt27bFsmXL0L9/f9ja\n2sLX1xepqamV7v/dd9/hnXfewYoVK6BUKhEZGQkfHx90794d7u7u6NevH77++mucO3cOPj4+AABr\na2tER0fD29sbw4cPh5eXF5ycnNCjR49q837++ed48cUX4eTkhE6dOukUmZubG3r27Im8vDz88MMP\nqFevHsaPHw+VSgVvb28IIWBnZ4fY2Nin9OhRTfDjGohIGl4iEZE0LBgikoYFQ0TSsGCISBoWDBFJ\nw4IhImlYMEQkDQuGiKT5P7hFlIscvnnzAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1185f0d50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    0.4s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model RidgeClassifier(alpha=1.0, class_weight=None, copy_X=True, fit_intercept=True,\n",
      "        max_iter=None, normalize=False, random_state=None, solver='auto',\n",
      "        tol=0.001)\n",
      "Grid Search Best Score: 0.735\n",
      "Grid Search Best Params: {'pca__n_components': 20}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.76      0.89      0.82       140\n",
      "          2       0.58      0.35      0.44        60\n",
      "\n",
      "avg / total       0.71      0.73      0.71       200\n",
      "\n",
      "Accuracy on Test Data: 0.730\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHrhJREFUeJzt3XlYVOXiB/DvMIDKErEpm6IikoLgVsoVU8ki8WpgaW6l\nXbO8CmZpqXG7LrndvJcWvaKSoqlX/VlILuklF0jTBAE1xA1iFIQQUxRElmHe3x8+znVCBJJ3Bobv\n53nmeZjznjnnO4Pz9T1nFhRCCAEiIglMDB2AiIwXC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWTDMy\nf/58tGnTBgqFAhs2bHjs7alUKigUCpw8efLxwzUBAwcORFhYmKFjNCkKvg/GsAoKCrBkyRLs2bMH\nubm5cHBwgK+vL8LDwxEcHNxg+0lPT0e3bt0QGxsLf39/2NjYoFWrVo+1zaqqKhQWFsLBwQGmpqYN\nlFR/VCoVOnTogOTkZPTu3bvW9W/cuAEzMzNYW1vrIZ1xaHr/KoyISqVCv379YG1tjaVLl8LPzw8a\njQYHDx7ElClTcOXKlQbbV2ZmJgAgJCQECoWiQbapVCrh5OTUINtqzCoqKmBubg47OztDR2l6BBnM\nkCFDhLOzsyguLq42duPGDe3Ply9fFiEhIcLKykpYWVmJ0NBQkZOTox2fN2+e8Pb2Flu3bhUdO3YU\nVlZW4qWXXhKFhYXacQA6FyGEmDBhghg6dKjOfu9v674zZ86IwMBAYW1tLaysrISvr684dOiQEEKI\n7OxsAUAkJydr109MTBTPPPOMaNGihWjdurWYMWOGKC8v144PGDBA/PWvfxVz584V9vb2wtHRUcyc\nOVNUVVXV+DjFxMQIS0tL8d133wkvLy/RqlUrMWzYMFFUVCR27NghOnXqJJ544gkxfvx4UVpaqr3d\nvn37REBAgHjyySeFra2teOGFF0RGRoZ2/PePyYABA3Qel2XLlglXV1fh6OiozT5t2jQhhBDnz58X\nFhYWYuPGjTr7MzMzE8ePH6/xvjQ3PAdjIDdu3MD+/fsRFhYGKyurauO2trYAACEEQkJCUFBQgEOH\nDuHw4cPIy8tDSEgIxANHtyqVCtu3b8fOnTsRHx+PtLQ0REREAABmzZqF6OhoAEB+fj7y8/PrnHPs\n2LFwdnZGUlIS0tLSMH/+fLRs2fKh6169ehVDhgxBjx49kJaWhnXr1mHr1q2YO3euznpbtmyBqakp\njh07hpUrV+Kzzz7D9u3bH5mjvLwc//rXv7BlyxYcPHgQJ0+exCuvvIKNGzfim2++QVxcHPbs2YNV\nq1Zpb3Pnzh3MmDEDSUlJSEhIgI2NDYYNG4aKigoAQFJSEgBg//79yM/PR2xsrPa2iYmJOHPmDPbv\n34+DBw9Wy+Pl5YVPP/0UYWFhyMrKQmFhISZOnIiIiAj07du3bg9uc2DohmuuTpw4IQCI2NjYR64X\nHx8vTExMRHZ2tnZZVlaWUCgU4vvvvxdC3Jt1tGjRQhQVFWnXWbRokfDw8NBe37Fjh/j9r7suMxhr\na2uxYcOGh2b7/Qzmww8/FB4eHjqzkZiYGGFubi7u3LkjhLg3C+jbt6/OdgYPHiwmTZpU42MQExMj\nAIjz589rl82cOVOYmJhoZ2k13Z8HlZSUCBMTE3HkyJGH5n9wOw4ODqKsrExn+YMzmPtCQkJEnz59\nRHBwsPjTn/4k1Gp1jftvjjiDMRBRx3Pr586dg4uLC9q3b69d1rFjR7i4uCAjI0O7zN3dHTY2Ntrr\nLi4uuHbt2mPnfO+99/Dmm28iMDAQixcvxvnz5x+Z1d/fHyYm//tnFRAQgIqKCu05IADw9fXVuV1d\nsrZo0QJeXl7a623atIGTkxMcHBx0lj24naysLIwdOxYeHh544okn0KZNG2g0mjqd2/Lx8UGLFi1q\nXe/LL79EZmYmfvjhB2zevBlKpbLW2zQnLBgD8fT0hEKhwLlz5x65nhCixpOyDy43MzOrNqbRaB65\nbRMTk2pFV1lZqXN9/vz5yMjIQEhICI4dOwZfX1+sX79e71l//yqVQqGodTvDhg1DYWEh1qxZgxMn\nTiAtLQ2mpqbaQ6RHsbS0rHUd4N6rc7du3UJZWRmuXr1ap9s0JywYA7Gzs0NQUBBWrlyJkpKSauNF\nRUUAgK5du+Lq1atQqVTasV9++QV5eXno2rXrY2VwdHSsdj7m1KlT1dbz9PTE9OnTsXfvXkyaNAlf\nfvnlQ7fXtWtXHD9+XOdJfvToUZibm8PDw+OxstbXb7/9hnPnzuHDDz/E4MGD0aVLFxQXF0OtVmvX\nMTc3B3Dv5fY/oqioCK+//jpmzZqFadOm4bXXXsPt27cbJL+xYMEY0KpVqyCEQO/evbFjxw5cuHAB\n58+fR1RUlPYwYvDgwfDz88O4ceOQkpKCkydPYty4cejZsycCAwMfa/+BgYFIS0vD+vXrkZmZiU8+\n+QQ//vijdvzu3buYNm0aEhISoFKpcOLECRw9erTGYps6dSry8vIwdepUnDt3Dnv37sWcOXMQFhYG\nCwuLx8paX7a2tnBwcEB0dDQyMzORmJiIKVOm6MyEWrdujVatWuG///0vCgoKcOvWrXrtY8qUKXBw\ncMDChQvxj3/8A9bW1pg2bVpD35UmjQVjQB06dEBqaiqef/55zJ49G76+vggMDMSuXbuwZs0aAPem\n/XFxcXB0dMTAgQMxaNAgODk5IS4u7rHfzxIUFIR58+YhIiICvXr1gkqlwtSpU7XjSqUSN2/exIQJ\nE+Dl5YXQ0FD4+/sjMjLyodtzdXXFvn37kJaWhu7du+Mvf/kLxowZgyVLljxWzj/CxMQE27dvx5kz\nZ+Dj44Np06bh448/1jmvYmpqii+++AJffvklXFxc8NJLL9V5+5s2bcKuXbuwZcsWmJmZoUWLFvjP\nf/6Dr7/+Glu3bpVxl5okvpOXiKThDIaIpGHBEJE0LBgikoYFQ0TSNIpPU5eVlSE9PR2Ojo58JyRR\nE3P/azt8fHyqfU6tURRMeno6xo0bZ+gYRPQYtmzZUu17dRpFwTg6OgK4F7A5fL9Ic9ShQwdDRyBJ\nTE1N0a5dO+3zWGfMAHmquX9Y5OTkBDc3NwOnIRkefIs+GaeHnd7gSV4ikoYFQ0TSsGCISBoWDBFJ\nw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQs\nGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4Ih\nImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBNJAffvgBw4cPh6urKxQKBTZs2KAdq6ysxOzZ\ns+Hr6wtLS0s4Oztj7NixuHLlis42Bg4cCIVCoXMZPXq0nu8J1VX//v3x7bffIjc3F0IITJgwQWc8\nJiYGQgidy/Hjxw2U1jBYMA2kpKQEPj4++Pzzz9GqVSudsdLSUqSmpiIiIgKpqan49ttvkZOTgxdf\nfBFqtVpn3TfeeAP5+fnay5o1a/R5N6gerKyskJ6ejnfeeQelpaUPXef777+Hk5OT9hIcHKznlIZl\naugAxiI4OFj7j2fixIk6YzY2Nvj+++91lq1Zswbe3t44d+4cunXrpl1uYWEBJycn6Xnp8e3btw/7\n9u0DAJ0Z64PKy8tRUFCgx1SNC2cwBnL79m0AgK2trc7ybdu2wcHBAd7e3pg1axaKi4sNEY8aSEBA\nAAoKCnDhwgWsXbsWjo6Oho6kV5zBGEBFRQVmzpyJYcOGwc3NTbt87NixcHd3h4uLC86ePYu5c+fi\n9OnT1WY/1DTs378fsbGxyM7ORvv27bFo0SIcOnQIvXr1QkVFhaHj6QULRs/UajXGjx+PoqIi7Nq1\nS2fsrbfe0v7crVs3dOzYEX369EFqaip69uyp76j0mLZv3679OT09HSkpKbh8+TKGDh2KnTt3GjCZ\n/kg9RNq8eTNGjBgBHx8fzJkzR+aumgS1Wo0xY8bgzJkzOHjwIOzt7R+5fu/evaFUKnHp0iU9JSSZ\n8vPzkZubC09PT0NH0RupM5jWrVtj6tSpOHLkCMrLy2XuqtGrrKzE6NGjkZ6ejoSEhDqdyP35559R\nVVUFZ2dnPSQk2ezt7eHq6or8/HxDR9EbqQXzwgsvALj3RDH2M+klJSXIzMwEAGg0Gly5cgWnTp2C\nnZ0dXFxcMHLkSCQnJ2P37t1QKBT49ddfAdx7halVq1bIysrCli1bEBwcDAcHB2RkZGDmzJno0aMH\n+vXrZ8i7RjWwtLREp06dAAAmJiZo164d/Pz8cOPGDdy4cQPz58/HN998g/z8fLRv3x5Lly7FtWvX\nms3hEQBA6EFkZKSYPXt2jeM5OTmic+fOIicnRx9xpDh8+LAAUO0yYcIEkZ2d/dAxACImJkYIIcSV\nK1fEs88+K+zs7IS5ubnw8PAQ06dPF7/99pth71gDqen+N+XLgAEDHnpfY2JiRMuWLcX+/ftFQUGB\nKC8vFyqVSsTExAg3NzeD527oi6mpaY3PX57kbSADBw7EvefRwz1qDADatm2LxMTEho5FEiUmJkKh\nUNQ4/uKLL+oxTePE98EQkTQsGCKSRuohklqtRlVVFTQaDaqqqlBeXg6lUglTUx6ZETUHUmcwUVFR\n8PX1xdq1a7Fr1y74+voiKipK5i6JqBGROpUIDw9HeHi4zF0QUSPGczBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVD\nRNKwYIhIGtOaBm7fvv3IGz7xxBMNHoaIjEuNBePt7Q2FQgEhhHbZ/esKhQJXrlzRS0AiarpqLJic\nnBx95iAiI1SnczDbtm3DkiVLAAC5ublISUmRGoqIjEOtBRMWFobDhw9j06ZNAAALCwtMmTJFejAi\navpqPES679ixY0hNTUWPHj0AAHZ2dqioqJAejIiavlpnMGZmZtBoNFAoFACA3377DSYmfHWbiGpX\na1NMmzYNL7/8MgoLCzFv3jwEBARg9uzZ+shGRE1crYdIr7/+Onr16oUDBw4AAHbs2AEfHx/pwYio\n6au1YACgqqoKZmZmUCgU0Gg0sjMRkZGo9RBp8eLFGDNmDPLy8pCbm4uxY8di6dKl+shGRE1crTOY\nzZs3IyUlBRYWFgCAiIgI9OrVC3PnzpUejoiatlpnMO7u7lCr1drrarUaHTt2lBqKiIxDjTOYd999\nFwqFAhYWFvD29kZQUBAUCgXi4+MREBCgz4xE1ETVWDD3Xyny9vbG0KFDtcv79u0rPxURGYUaC2bS\npEn6zEFERqjWk7xZWVmIiIhARkYGysrKtMsvXrwoNRgRNX21nuSdOHEi3njjDQghsG/fPowaNQqj\nR4/WRzYiauJqLZjS0lIEBQUBADw8PLBo0SIcPnxYejAiavpqPURq0aIFhBDw8PDA6tWr4erqimvX\nrukjGxE1cbUWzKeffoqSkhJ88cUXiIiIwK1bt7B+/Xp9ZCOiJq7WgunTpw8AwNraWvulU0REdVFj\nwYSGhmq/A+ZhYmNjpQQiIuNRY8GEhYXpMwcAYOjQcTAxqdMHvKmJsbS0MXQEkkSprPm1ohqfzc89\n95yUMETUfPC7L4lIGhYMEUlT54IpLy+XmYOIjFCtBZOUlIRu3brB09MTAHD69GmEh4dLD0ZETV+t\nBTN9+nTs2bMH9vb2AAA/Pz9+VICI6qTWgtFoNHB3d9dZplQqpQUiIuNR65tO2rZti6SkJCgUClRV\nVWHFihXo3LmzPrIRURNX6wwmKioKkZGRuHLlCtq0aYOffvoJUVFR+shGRE1crTOY1q1bY9u2bfrI\nQkRGptaCmTx58kM/k7R27VopgYjIeNRaMIMHD9b+XFZWhp07d6Jt27ZSQxGRcai1YF599VWd66+9\n9hqef/55aYGIyHjU+6MC2dnZuHz5sowsRGRkap3B2Nraas/BaDQa2NnZYdmyZdKDEVHT98iCEULg\n9OnTcHV1BQCYmJg88kuoiIge9MhDJIVCgdDQUCiVSiiVSpYLEdVLredgnnnmGaSmpuojCxEZmRoP\nkdRqNUxNTXH06FFER0fDw8MDlpaWEEJAoVCwdIioVjUWzP2ZS1xcnD7zEJERqbFghBAA7v01RyKi\nP6LGgiksLERkZGSNN3zvvfekBCIi41FjwVRVVaGkpEQ7kyEiqq8aC8bZ2Rl///vf9ZmFiIxMjS9T\nc+ZCRI+rxoI5ePCgPnMQkRGqsWDs7Oz0mYOIjBD/8BoRScOCISJpWDBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgJBk16iVs3x6NI0d24ciRXdi4cQUCAvpox+3sbLFgwQeI\nj9+OY8f2YuXKpWjXztWAiak+Zs58F4mJh5CXdwUqVSb+7/+2oWvXLjrrDB8+DHFx30ClykRJSRH6\n9w8wUFrDYcFIcu1aIb74Ihpjx07BuHFTkZSUhsjIhfD07AgA+PTThWjXzhXvvvt3jBnzNvLzr2H1\n6uVo2bKlgZNTXfTvH4Do6HUYPDgIQ4cOR1WVGrt3x8HW9kntOhYWFjhxIglz5kQYMKlhmRo6gLFK\nSDimc/3f/16PkSOHwde3K8rLK+Dr2xWvvjoZFy/+AgBYsuQzHDiwA0OGBGLnzu8MEZnqISTkZZ3r\nb775NvLyrqBv377Yt28/AGDbtu0AAHv75vtXUjmD0QMTExMEBQ2ChUUrnD59FubmZgCA8vIK7TpC\nCFRUVKJ7dx9DxaTHYGVlBaVSiaKiIkNHaVQ4g5GoU6cO2LhxBczNzXH37l289948ZGZmw9RUiby8\nXxEePgkLF0aitPQuxo9/BU5OreHg0Hz/t2vKli9fhtOnz+DEiSRDR2lUOIORSKXKwejRb2HChDDs\n2LELCxfOhodHe6jVVZg1az7c3FyQmBiH48e/Q+/efjh69AQ0Go2hY1M9LV26GP7+fTFu3Gv8/f2O\ntBlMRUUF5s+fj+PHj6OoqAju7u549913MWDAAFm7bHTUajVycvIAABkZF+Ht7YXx41/BggX/xLlz\nlzB69NuwsrKEmZkpbt68ha++WomMjIsGTk31sWzZErzyyggEBw+DSnXZ0HEaHWkzGLVaDWdnZ2za\ntAkpKSl45513MGPGDOTm5sraZaOnUJjAzMxMZ1lJyR3cvHkL7dq5omvXzkhI+NFA6ai+PvlkGUaN\negVDhw7HxYuXDB2nUZI2g7GwsEB4eLj2+qBBg+Dm5oazZ8/Czc1N1m4bjenT38SRIyfw66/XYGlp\ngSFDAtG7tx+mT7/3kuXgwc+iqOg28vML4OnZAe+/Pw0JCT/ip59SDJyc6iIycjlGj34VY8aMx82b\nRWjdujUA4M6dO7hz5w4AwNb2Sbi5tcWTT9oAADp27ICiolsoKCjAtWvXDJZdn/R2kvf69etQqVTo\n1KmTvnZpUPb2dli8eC7s7W1RUnIHly79grCwuTh+/CQAwNHRHjNn/hX29ra4fv0G9uyJx9q1mw2c\nmurqrbcmAwD27t2ls3zJkmVYsmQZACA4OBhr1qzSjv373yuqrWPsFEIIIXsnlZWVmDx5Mtq1a4eF\nCxdWG8/NzcVzzz2Hli2dYGLCF7aM0aVLnJkZK6XSBE5Ojjh48GC1oxPpryJpNBp88MEHMDMzw0cf\nfSR7d0TUiEidLgghEBERgevXryM6OrraCU4iMm5SC2bevHnIyspCTEwMP2ND1AxJK5irV69i+/bt\nMDc3R0DA/z5FumDBAgwfPlzWbomoEZFWMK6urrhw4YKszRNRE8CPChCRNCwYIpKGBUNE0rBgiEga\nFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHB\nEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwR\nScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0\nLBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaU0MHAICq\nqioAgEajNnASkkWp5P9lxur+7/b+8/hBjaJgCgsLAQAVFdcNnIRkcXJyNHQEkqywsBDu7u46yxRC\nCGGgPFplZWVIT0+Ho6MjlEqloeMQUT1UVVWhsLAQPj4+aNmypc5YoygYIjJOPDAmImlYMEQkDQuG\niKRhwRCRNCwYPdu8eTNGjBgBHx8fzJkzx9BxqAFVVFTgww8/xKBBg9CjRw+EhIQgMTHR0LEMqlG8\nD6Y5ad26NaZOnYojR46gvLzc0HGoAanVajg7O2PTpk1wcXFBYmIiZsyYgd27d8PNzc3Q8QyCBaNn\nL7zwAgDg559/RkFBgYHTUEOysLBAeHi49vqgQYPg5uaGs2fPNtuC4SESkSTXr1+HSqVCp06dDB3F\nYFgwRBJUVlZi1qxZCA0NhYeHh6HjGAwLhqiBaTQafPDBBzAzM8NHH31k6DgGxXMwRA1ICIGIiAhc\nv34d0dHRMDMzM3Qkg2LB6JlarUZVVRU0Gg2qqqpQXl4OpVIJU1P+KozBvHnzkJWVhZiYmGof/GuO\n+GFHPVuxYgVWrlypsywsLEzn1Qdqmq5evYrAwECYm5vr/IexYMECDB8+3IDJDIcFQ0TS8CQvEUnD\ngiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFY6SUSiW6d+8OHx8fjBw5EqWlpX94WwkJCfjzn/8MANi1\naxeWLVtW47pFRUVYtWpVvfcxf/58/POf/6zz8gdNnDgRX3/9dZ33pVKp4OPjU++MVH8sGCPVqlUr\nnDp1Cunp6TA3N8fq1at1xoUQ0Gg09d7u8OHDH/lFWX+0YMg4sWCagf79+yMzMxMqlQpdunTB1KlT\n0bNnT+Tk5CA+Ph7+/v7o2bMnRo4ciZKSEgDA/v378dRTTyEgIACxsbHabW3YsAFhYWEAgIKCAoSG\nhsLPzw9+fn44duwY5syZg6ysLHTv3h3vv/8+AGD58uV4+umn4evri3nz5mm3tXjxYnh5eWHw4MG4\ncOFCrfcjOjoaTz/9NPz8/PDyyy/rzMoOHDiA/v37o3PnztizZw+Ae3+v5/3339fue82aNY//YFK9\nsGCMnFqtxr59+9CtWzcAwIULF/D6668jLS0NlpaWWLRoEQ4cOIDU1FT07t0bkZGRKCsrw+TJk7F7\n924cOXIEv/7660O3PX36dAwYMACnT59GamoqvL29sWzZMnh4eODUqVNYvnw54uPjcenSJSQlJeHU\nqVNISUnBDz/8gJSUFGzbtg1paWmIjY1FcnJyrfdlxIgRSE5OxunTp9GlSxesW7dOO6ZSqZCYmIi9\ne/diypQpKCsrw7p162BjY4Pk5GQkJycjOjoa2dnZDfPAUp3wE3ZG6u7du+jevTuAezOYSZMmIS8v\nD+7u7ujbty8A4KeffkJGRgb69esH4N53yvr7++P8+fPo0KEDPD09AQDjx4/H2rVrq+3j0KFD+Oqr\nrwDcO+djY2ODmzdv6qwTHx+P+Ph49OjRAwBQUlKCS5cuobi4GKGhobCwsACAOn1WJz09HX/7299Q\nVFSEkpISBAUFacdGjRoFExMTeHp6omPHjjh//jzi4+Nx5swZ7fmZW7du4dKlS+jcuXPdH0h6LCwY\nI3X/HMzvWVpaan8WQuD555/H1q1bddY5deoUFApFg+QQQmDu3Ll4++23dZZ/9tln9d7HxIkTERcX\nBz8/P2zYsAEJCQnasd9vS6FQQAiBFStW6BQRcG+2Q/rBQ6RmrG/fvvjxxx+RmZkJACgtLcXFixfx\n1FNPITs7G1lZWQBQrYDue+655xAVFQXg3vmO27dvw9raGsXFxdp1goKCsH79eu25natXr+LatWt4\n9tlnsXPnTty9exfFxcXYvXt3rXmLi4vh7OyMyspKbNmyRWdsx44d0Gg0yMrKwi+//AIvLy8EBQUh\nKioKlZWVAICLFy/izp079XyU6HFwBtOMOTo6YsOGDRgzZoz2LxwsWrQInTt3xtq1azF06FA4ODgg\nICAA6enp1W7/+eef46233sK6deugVCoRFRUFf39/9OvXDz4+PhgyZAiWL1+Oc+fOwd/fHwBgZWWF\nzZs3o2fPnnj11VfRvXt3uLu7o3///rXm/fjjj9GnTx+4u7ujW7duOkXm5eWFAQMGoKCgAKtXr0bL\nli3x5ptvQqVSoWfPnhBCwNHREXFxcQ306FFd8OsaiEgaHiIRkTQsGCKShgVDRNKwYIhIGhYMEUnD\ngiEiaVgwRCQNC4aIpPl/90wUirL96NwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1174736d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 7 candidates, totalling 35 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  35 out of  35 | elapsed:    0.4s finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model BernoulliNB(alpha=1.0, binarize=0.0, class_prior=None, fit_prior=True)\n",
      "Grid Search Best Score: 0.7175\n",
      "Grid Search Best Params: {'pca__n_components': 8}\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.74      0.94      0.83       140\n",
      "          2       0.64      0.23      0.34        60\n",
      "\n",
      "avg / total       0.71      0.73      0.68       200\n",
      "\n",
      "Accuracy on Test Data: 0.730\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHjZJREFUeJzt3Xl4TOfiB/DvZLLIZkkTspFLEEskEW3JFUUsqbY06aKl\nqFbdq1kU1VpSv6ityr2qpYIgaVFcLblCaVCCBlESpLElTEOiEdWQRbaZ9/eHx1zTiFB552T5fp5n\nnsecc+ac70zkm/e8s6mEEAJERBKYKB2AiOovFgwRScOCISJpWDBEJA0LhoikYcEQkTQsmAZk5syZ\naNGiBVQqFWJjYx97fxqNBiqVCj///PPjh6sD+vTpg7CwMKVj1Ckqvg5GWbm5uZg3bx62b9+OK1eu\nwN7eHl5eXggPD8dzzz1XY8dJS0tDly5dsGXLFvj5+aFJkyawtLR8rH1qtVrk5eXB3t4epqamNZTU\neDQaDVq3bo1jx47hySefrHb7GzduwMzMDLa2tkZIVz/Uvf8V9YhGo0HPnj1ha2uLTz75BN7e3tDp\ndNi7dy/GjRuHrKysGjtWRkYGACAoKAgqlapG9qlWq+Ho6Fgj+6rNysrKYG5uDjs7O6Wj1D2CFDNo\n0CDh5OQkCgoKKq27ceOG/t+//vqrCAoKEjY2NsLGxkYEBweLy5cv69dHRkaKzp07iw0bNog2bdoI\nGxsb8eKLL4q8vDz9egAGFyGEePPNN8Xzzz9vcNy7+7rr1KlTIiAgQNja2gobGxvh5eUlfvzxRyGE\nEJcuXRIAxLFjx/TbJyYmiqefflpYWFiI5s2biwkTJojS0lL9+t69e4t3331XTJs2TTzxxBPCwcFB\nvP/++0Kr1Vb5OMXExAhra2vx/fffCw8PD2FpaSkGDx4s8vPzxebNm0Xbtm1F48aNxYgRI0RxcbH+\ndjt37hT+/v6iadOmolmzZmLgwIEiPT1dv/7Pj0nv3r0NHpf58+cLFxcX4eDgoM8eGhoqhBDi7Nmz\nwsrKSnz11VcGxzMzMxOHDx+u8r40NJyDUciNGzewa9cuhIWFwcbGptL6Zs2aAQCEEAgKCkJubi5+\n/PFH7Nu3Dzk5OQgKCoK45+xWo9Fg06ZN2Lp1KxISEpCSkoKIiAgAwOTJkxEdHQ0AuHr1Kq5evfrQ\nOYcPHw4nJyckJycjJSUFM2fORKNGje67bXZ2NgYNGoSuXbsiJSUFq1evxoYNGzBt2jSD7davXw9T\nU1MkJSVh6dKlWLx4MTZt2vTAHKWlpfj3v/+N9evXY+/evfj555/xyiuv4KuvvsJ3332HuLg4bN++\nHcuWLdPfpqioCBMmTEBycjL279+PJk2aYPDgwSgrKwMAJCcnAwB27dqFq1evYsuWLfrbJiYm4tSp\nU9i1axf27t1bKY+Hhwc+++wzhIWFITMzE3l5eRg9ejQiIiLQo0ePh3twGwKlG66hOnr0qAAgtmzZ\n8sDtEhIShImJibh06ZJ+WWZmplCpVGL37t1CiDujDgsLC5Gfn6/fZs6cOcLd3V1/ffPmzeLPP+6H\nGcHY2tqK2NjY+2b78whm+vTpwt3d3WA0EhMTI8zNzUVRUZEQ4s4ooEePHgb76d+/vxgzZkyVj0FM\nTIwAIM6ePatf9v777wsTExP9KK2q+3OvwsJCYWJiIg4ePHjf/Pfux97eXpSUlBgsv3cEc1dQUJDo\n3r27eO6558Tf//53UVFRUeXxGyKOYBQiHnJu/cyZM3B2dsbf/vY3/bI2bdrA2dkZ6enp+mVubm5o\n0qSJ/rqzszOuXbv22DknTZqEd955BwEBAZg7dy7Onj37wKx+fn4wMfnffyt/f3+UlZXp54AAwMvL\ny+B2D5PVwsICHh4e+ustWrSAo6Mj7O3tDZbdu5/MzEwMHz4c7u7uaNy4MVq0aAGdTvdQc1uenp6w\nsLCodrtVq1YhIyMDBw4cwLp166BWq6u9TUPCglFIu3btoFKpcObMmQduJ4SoclL23uVmZmaV1ul0\nugfu28TEpFLRlZeXG1yfOXMm0tPTERQUhKSkJHh5eWHNmjVGz/rnZ6lUKlW1+xk8eDDy8vKwYsUK\nHD16FCkpKTA1NdWfIj2ItbV1tdsAd56du3nzJkpKSpCdnf1Qt2lIWDAKsbOzQ2BgIJYuXYrCwsJK\n6/Pz8wEAnTp1QnZ2NjQajX7dxYsXkZOTg06dOj1WBgcHh0rzMampqZW2a9euHcaPH48dO3ZgzJgx\nWLVq1X3316lTJxw+fNjgl/zQoUMwNzeHu7v7Y2V9VL///jvOnDmD6dOno3///ujYsSMKCgpQUVGh\n38bc3BzAnafb/4r8/HyMGjUKkydPRmhoKEaOHIlbt27VSP76ggWjoGXLlkEIgSeffBKbN2/GuXPn\ncPbsWURFRelPI/r37w9vb2+88cYbOH78OH7++We88cYb8PX1RUBAwGMdPyAgACkpKVizZg0yMjKw\nYMEC/PTTT/r1t2/fRmhoKPbv3w+NRoOjR4/i0KFDVRZbSEgIcnJyEBISgjNnzmDHjh2YOnUqwsLC\nYGVl9VhZH1WzZs1gb2+P6OhoZGRkIDExEePGjTMYCTVv3hyWlpb44YcfkJubi5s3bz7SMcaNGwd7\ne3vMmjULn376KWxtbREaGlrTd6VOY8EoqHXr1jhx4gQGDBiAKVOmwMvLCwEBAdi2bRtWrFgB4M6w\nPy4uDg4ODujTpw/69u0LR0dHxMXFPfbrWQIDAxEZGYmIiAh069YNGo0GISEh+vVqtRp//PEH3nzz\nTXh4eCA4OBh+fn5YtGjRfffn4uKCnTt3IiUlBT4+Pnj77bcxbNgwzJs377Fy/hUmJibYtGkTTp06\nBU9PT4SGhmL27NkG8yqmpqb44osvsGrVKjg7O+PFF1986P2vXbsW27Ztw/r162FmZgYLCwt88803\n+Pbbb7FhwwYZd6lO4it5iUgajmCISBoWDBFJw4IhImlYMEQkTa14N3VJSQnS0tLg4ODAV0IS1TF3\nP7bD09Oz0vvUakXBpKWl4Y033lA6BhE9hvXr11f6XJ1aUTAODg4A7gRsCJ8v0hC1bt1a6Qgkiamp\nKVq1aqX/PTZYp0CeSu6eFjk6OsLV1VXhNCTDvS/Rp/rpftMbnOQlImlYMEQkDQuGiKRhwRCRNCwY\nIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEi\naVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKG\nBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgw\nRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWTA05cOAAhgwZAhcXF6hUKsTGxhqsnzFjBjp06ABr\na2s0a9YM/fr1Q1JSkn79jRs3EB4ejg4dOsDS0hItW7bEu+++i99//93I94T+ChMTE8yaNQsXL17E\n7du3cfHiRcyePRtqtVrpaIpiwdSQwsJCeHp64vPPP4elpWWl9R4eHvjyyy9x+vRpHDp0CK1bt8az\nzz6L3NxcAEBOTg6ys7OxYMECnD59GuvWrcOBAwcwbNgwY98V+gumTJmC0NBQjB8/Hh06dMB7772H\n0NBQTJs2TeloyhK1wOXLl0X79u3F5cuXlY5SI6ytrUVMTMwDt7l586YAIHbt2lXlNjt27BAqlUrc\nvHmzhhMaH4B6fYmPjxexsbEGy2JjY0V8fLzi2WRfTE1Nq/z95QhGAWVlZVi5ciUaN24MHx+fKre7\ndesWLCwsYGVlZcR09FccOnQIffv2hYeHBwCgY8eOCAgIwPfff69wMmWZKh2gIdm+fTtef/11FBcX\nw8nJCbt370aLFi3uu21+fj5mzJiBsWPHwtSUP6ba7tNPP4WtrS3S09Oh1WphZmaGOXPmICoqSulo\niuIIxoj69u2L1NRUJCUl4dlnn8XQoUNx9erVStsVFRVh8ODBcHFxwYIFCxRISo/qtddew6hRozB8\n+HD4+vpi5MiRCAkJwdtvv610NGXJPO9eu3atCA4OFp07dxZTpkypcruGOAcjhBBt27YVs2bNMlhW\nUFAgevXqJfz9/UVBQYGkhMaHWjBXIPOSlZUlxo8fb7AsIiJCXLhwQfFssi8PmoOROvZu3rw5QkJC\ncPDgQZSWlso8VJ2k0+kMHpeCggIMGjQIQgjs2rULNjY2CqajR2FlZQWtVmuwTKvVwsSkYZ8kSC2Y\ngQMHAgBOnz6tfzq2viosLERGRgaAO8WRlZWF1NRU2NnZoWnTpliwYAEGDx4MJycn5OXl4csvv8SV\nK1cwdOhQAHfKZeDAgbh16xbi4uJQVFSEoqIiAICdnR3Mzc0Vu29Uvfj4eEydOhWXLl3CL7/8gq5d\nu2LSpEn4+uuvlY6mLGMMjxctWlTvT5H27dt33+Hjm2++KYqKikRQUJBwcnIS5ubmwsnJSQwZMkQc\nOXKk2tsDEPv27VPujtWQqu5bfbnY2NiIzz77TGg0GlFcXCwyMzPF3LlzhYWFheLZZF8UO0VqSPr0\n6YM7v0f3t3Xr1se6PdVuhYWFmDhxIiZOnKh0lFqlYZ8gEpFULBgikkbqKVJFRQW0Wi10Oh20Wi1K\nS0uhVqv5wjGiBkLqCCYqKgpeXl5YuXIltm3bBi8vrwb/ykaihkTqUCI8PBzh4eEyD0FEtRjnYIhI\nGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRh\nwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYM\nEUnDgiEiaVgwRCQNC4aIpGHBEJE0plWtuHXr1gNv2Lhx4xoPQ0T1S5UF07lzZ6hUKggh9MvuXlep\nVMjKyjJKQCKqu6osmMuXLxszBxHVQw81B7Nx40bMmzcPAHDlyhUcP35caigiqh+qLZiwsDDs27cP\na9euBQBYWVlh3Lhx0oMRUd1X5SnSXUlJSThx4gS6du0KALCzs0NZWZn0YERU91U7gjEzM4NOp4NK\npQIA/P777zAx4bPbRFS9apsiNDQUL7/8MvLy8hAZGQl/f39MmTLFGNmIqI6r9hRp1KhR6NatG/bs\n2QMA2Lx5Mzw9PaUHI6K6r9qCAQCtVgszMzOoVCrodDrZmYionqj2FGnu3LkYNmwYcnJycOXKFQwf\nPhyffPKJMbIRUR1X7Qhm3bp1OH78OKysrAAAERER6NatG6ZNmyY9HBHVbdWOYNzc3FBRUaG/XlFR\ngTZt2kgNRUT1Q5UjmIkTJ0KlUsHKygqdO3dGYGAgVCoVEhIS4O/vb8yMRFRHVVkwd58p6ty5M55/\n/nn98h49eshPRUT1QpUFM2bMGGPmIKJ6qNpJ3szMTERERCA9PR0lJSX65efPn5cajIjqvmoneUeP\nHo233noLQgjs3LkTQ4cOxeuvv26MbERUx1VbMMXFxQgMDAQAuLu7Y86cOdi3b5/0YERU91V7imRh\nYQEhBNzd3bF8+XK4uLjg2rVrxshGRHVctQXz2WefobCwEF988QUiIiJw8+ZNrFmzxhjZiKiOq7Zg\nunfvDgCwtbXVf+gUEdHDqLJggoOD9Z8Bcz9btmyREoiI6o8qCyYsLMyYOQAAI0e+D1PTRkY/Lsln\nZcWvuamv1Gp1leuqLJh+/fpJCUNEDQc/+5KIpGHBEJE0D10wpaWlMnMQUT1UbcEkJyejS5cuaNeu\nHQDg5MmTCA8Plx6MiOq+agtm/Pjx2L59O5544gkAgLe3N98qQEQPpdqC0el0cHNzM1j2oKeliIju\nqvaVvC1btkRycjJUKhW0Wi2WLFmC9u3bGyMbEdVx1Y5goqKisGjRImRlZaFFixY4cuQIoqKijJGN\niOq4akcwzZs3x8aNG42RhYjqmWoLZuzYsfd9T9LKlSulBCKi+qPagunfv7/+3yUlJdi6dStatmwp\nNRQR1Q/VFsxrr71mcH3kyJEYMGCAtEBEVH888lsFLl26hF9//VVGFiKqZ6odwTRr1kw/B6PT6WBn\nZ4f58+dLD0ZEdd8DC0YIgZMnT8LFxQUAYGJi8sAPoSIiutcDT5FUKhWCg4OhVquhVqtZLkT0SKqd\ng3n66adx4sQJY2QhonqmylOkiooKmJqa4tChQ4iOjoa7uzusra0hhIBKpWLpEFG1qiyYuyOXuLg4\nY+YhonqkyoIRQgC4822ORER/RZUFk5eXh0WLFlV5w0mTJkkJRET1R5UFo9VqUVhYqB/JEBE9qioL\nxsnJCf/3f/9nzCxEVM9U+TQ1Ry5E9LiqLJi9e/caMwcR1UNVFoydnZ0xcxBRPcQvXiMiaVgwRCQN\nC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBg\niEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhaMkQwb9gJ27/4KYWEjDZa7\nuLRAZGQ4tm5dhvj4lVi27GO0auWkUEp6FD17/h3/+c8GXLhwBkVFNzFixPAqt12y5HMUFd3Ee++F\nGzGh8qr8bmqqOR07umPQoD7IzMwyWO7oaI/Fiz/Cnj0/4YMPtqGwsBitWjnh9u1ShZLSo7CxsUZ6\n+hl8881GREcvr3K7oKAX0a2bL3JycoyYrnZgwUhmZWWJqVPHYdGi1Rgx4kWDdW+99QqOH0/DihUb\n9ct++y3P2BHpL/rhh9344YfdAIAVK5bdd5uWLVti4cL5eOGFF7F167fGjFcr8BRJsokT38LBg8eQ\nmnrGYLlKpUKPHj7IysrBvHnvY/PmJVi6NBK9ez+tUFKqaWq1GrGxq/Hpp//CuXPnlY6jCBaMRIMG\n9YazcwvExn5XaV3Tpo1hZWWJ118fjOPH0zB16kLs23cE06aNQ/fu3gqkpZr20UfTcePGDaxatVrp\nKIrhKZIkrq6OePvtVzBp0jxUVGgrrTcxUQEADh8+ge+++wEAkJmZhfbtW2PIkP44evSkUfNSzfL3\n74kRI4bDz89f6SiKklYwZWVlmDlzJg4fPoz8/Hy4ublh4sSJ6N27t6xD1iqdOrVF06aNER09V79M\nrVajSxcPvPBCXwwe/A9UVFTg118NJ/6ysnLQp08PY8elGvbMM73g6OiIzMz/nRqZmppi9uyPERr6\nLtq376RgOuORVjAVFRVwcnLC2rVr4ezsjMTEREyYMAHx8fFwdXWVddha46efTuD8+ekGyyZPfgfZ\n2bnYsCEe5eUVOHfuElq2dDTYxtXVEdeuXTdmVJIgOnoV4uL+a7Dsv//dgs2bv0NMTKwyoRQgrWCs\nrKwQHv6/5/z79u0LV1dX/PLLLw2iYIqKilFUVGywrKSkFAUFRdBosgEA//nP9/joo1CcPn0eqanp\n8PbuiD59uiMy8gslItMjsra2hrt7GwCAiYkJXF1d4eXVBTdu/IErV64gL8/wD0V5eTlyc3Nx4UKG\nEnEVYbRJ3uvXr0Oj0aBt27bGOmStl5R0AosXx+DVVwdh5cq5CAoagAULopGczPmXusDXtysOHz6E\nw4cPwcrKCjNmRODw4UOYMWN69TduIFRCCCH7IOXl5Rg7dixatWqFWbNmVVp/5coV9OvXD87OPjA1\nbSQ7DikgKSlO6QgkiVqthpOTA/bu3Vvp7ET6CEan0+HDDz+EmZkZZsyYIftwRFSLSH2aWgiBiIgI\nXL9+HdHR0TAzM5N5OCKqZaQWTGRkJDIzMxETE4NGjXjqQ9TQSCuY7OxsbNq0Cebm5vD3/9+LjT7+\n+GMMGTJE1mGJqBaRVjAuLi44d+6crN0TUR3A9yIRkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgw\nRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE\n0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQN\nC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBg\niEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGlOlAwCAVqsFAFRUlCqchGRR\nq9VKRyBJ1Oo745S7v8f3qhUFk5eXBwC4du2MwklIFicnB6UjkGR5eXlwc3MzWKYSQgiF8uiVlJQg\nLS0NDg4O/EtHVMdotVrk5eXB09MTjRo1MlhXKwqGiOonTvISkTQsGCKShgVDRNKwYIhIGhaMka1b\ntw4vvfQSPD09MXXqVKXjUA0qKyvD9OnT0bdvX3Tt2hVBQUFITExUOpaiasXrYBqS5s2bIyQkBAcP\nHkRpKV9YWJ9UVFTAyckJa9euhbOzMxITEzFhwgTEx8fD1dVV6XiKYMEY2cCBAwEAp0+fRm5ursJp\nqCZZWVkhPDxcf71v375wdXXFL7/80mALhqdIRJJcv34dGo0Gbdu2VTqKYlgwRBKUl5dj8uTJCA4O\nhru7u9JxFMOCIaphOp0OH374IczMzDBjxgyl4yiKczBENUgIgYiICFy/fh3R0dEwMzNTOpKiWDBG\nVlFRAa1WC51OB61Wi9LSUqjVapia8kdRH0RGRiIzMxMxMTGV3vjXEPHNjka2ZMkSLF261GBZWFiY\nwbMPVDdlZ2cjICAA5ubmBn8wPv74YwwZMkTBZMphwRCRNJzkJSJpWDBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYOoptVoNHx8feHp64tVXX0VxcfFf3tf+/fvxwgsvAAC2bduG+fPnV7ltfn4+li1b9sjH\nmDlzJv71r3899PJ7jR49Gt9+++1DH0uj0cDT0/ORM9KjY8HUU5aWlkhNTUVaWhrMzc2xfPlyg/VC\nCOh0ukfe75AhQx74QVl/tWCofmLBNAC9evVCRkYGNBoNOnbsiJCQEPj6+uLy5ctISEiAn58ffH19\n8eqrr6KwsBAAsGvXLnTo0AH+/v7YsmWLfl+xsbEICwsDAOTm5iI4OBje3t7w9vZGUlISpk6diszM\nTPj4+OCDDz4AACxcuBBPPfUUvLy8EBkZqd/X3Llz4eHhgf79++PcuXPV3o/o6Gg89dRT8Pb2xssv\nv2wwKtuzZw969eqF9u3bY/v27QDufF/PBx98oD/2ihUrHv/BpEfCgqnnKioqsHPnTnTp0gUAcO7c\nOYwaNQopKSmwtrbGnDlzsGfPHpw4cQJPPvkkFi1ahJKSEowdOxbx8fE4ePAgfvvtt/vue/z48ejd\nuzdOnjyJEydOoHPnzpg/fz7c3d2RmpqKhQsXIiEhARcuXEBycjJSU1Nx/PhxHDhwAMePH8fGjRuR\nkpKCLVu24NixY9Xel5deegnHjh3DyZMn0bFjR6xevVq/TqPRIDExETt27MC4ceNQUlKC1atXo0mT\nJjh27BiOHTuG6OhoXLp0qWYeWHoofIddPXX79m34+PgAuDOCGTNmDHJycuDm5oYePXoAAI4cOYL0\n9HT07NkTwJ3PlPXz88PZs2fRunVrtGvXDgAwYsQIrFy5stIxfvzxR3z99dcA7sz5NGnSBH/88YfB\nNgkJCUhISEDXrl0BAIWFhbhw4QIKCgoQHBwMKysrAHio9+qkpaXho48+Qn5+PgoLCxEYGKhfN3To\nUJiYmKBdu3Zo06YNzp49i4SEBJw6dUo/P3Pz5k1cuHAB7du3f/gHkh4LC6aeujsH82fW1tb6fwsh\nMGDAAGzYsMFgm9TUVKhUqhrJIYTAtGnT8M9//tNg+eLFix/5GKNHj0ZcXBy8vb0RGxuL/fv369f9\neV8qlQpCCCxZssSgiIA7ox0yDp4iNWA9evTATz/9hIyMDABAcXExzp8/jw4dOuDSpUvIzMwEgEoF\ndFe/fv0QFRUF4M58x61bt2Bra4uCggL9NoGBgVizZo1+bic7OxvXrl3DM888g61bt+L27dsoKChA\nfHx8tXkLCgrg5OSE8vJyrF+/3mDd5s2bodPpkJmZiYsXL8LDwwOBgYGIiopCeXk5AOD8+fMoKip6\nxEeJHgdHMA2Yg4MDYmNjMWzYMP03HMyZMwft27fHypUr8fzzz8Pe3h7+/v5IS0urdPvPP/8c//jH\nP7B69Wqo1WpERUXBz88PPXv2hKenJwYNGoSFCxfizJkz8PPzAwDY2Nhg3bp18PX1xWuvvQYfHx+4\nubmhV69e1eadPXs2unfvDjc3N3Tp0sWgyDw8PNC7d2/k5uZi+fLlaNSoEd555x1oNBr4+vpCCAEH\nBwfExcXV0KNHD4Mf10BE0vAUiYikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCTN/wPt\nG14XUB+vkwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1174a4e90>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for model in models:\n",
    "    pipeline = Pipeline([\n",
    "        ('ss', StandardScaler()),\n",
    "        ('pca', PCA(random_state=42)),\n",
    "        ('clf', model), \n",
    "    ])\n",
    "    grid_search = GridSearchCV(pipeline, parameters, verbose=1, cv=5)\n",
    "    grid_search.fit(data_train, label_train)\n",
    "    print \"Model\", model\n",
    "    print \"Grid Search Best Score:\", grid_search.best_score_\n",
    "    print \"Grid Search Best Params:\", grid_search.best_params_\n",
    "    \n",
    "    predictions = grid_search.best_estimator_.predict(data_test)\n",
    "    print \"Classification Report:\", classification_report(label_test, predictions)\n",
    "    print 'Accuracy on Test Data: {0:.3f}'.format(accuracy_score(label_test, predictions))  \n",
    "\n",
    "    cnf_matrix = confusion_matrix(label_test, predictions)\n",
    "    plot_confusion_matrix(cnf_matrix, classes=grid_search.best_estimator_.classes_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The grid search revealed SVC, Logistic Regression and Random Forest to be the best classifiers based on the grid searched best score on the training data. The number of principal components ranged from 4 to 25.\n",
    "\n",
    "Choosing the three best performing models, let's see if we can improve these models scores by tuning the hyperparameters of each of these classification algorithms along with PCA components to see if we can improve the score."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gridsearching Random Forest w/ PCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rf = RandomForestClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "rf_parameters = {\n",
    "           \"clf__n_estimators\"      : [100, 300],\n",
    "           # NUMBER OF TREES IN THE FOREST\n",
    "    \n",
    "           \"clf__criterion\"         : [\"gini\", \"entropy\"],\n",
    "           # METRICS ON WHICH TO SPLIT THE DATA:\n",
    "           # ENTROPY: MEASUREMENT OF INFORMATION GAIN\n",
    "           # GINI: PROBABILITY OF CLASSIFYING RANDOM SAMPLE CORRECTLY\n",
    "    \n",
    "           \"clf__max_depth\"         : [10, 20],\n",
    "           # MAX NO. OF FEATURES TO CONSIDER WHEN LOOKING FOR BEST SPLIT \n",
    "    \n",
    "           \"clf__min_samples_split\" : [2, 4],\n",
    "           # MINIMUM NUMBER OF SAMPLES REQUIRED TO SPLIT A NODE\n",
    "    \n",
    "           \"clf__bootstrap\"         : [True, False],\n",
    "           # WHETHER TO USE BOOTSTRAP SAMPLES OR NOT (FORM OF RESAMPLING)\n",
    "    \n",
    "           \"pca__n_components\"      : [2, 8, 10, 20, 25, 29]\n",
    "           # NO OF PRINCIPAL COMPONENTS\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 251,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 128 candidates, totalling 640 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done 640 out of 640 | elapsed: 14.0min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ranfom Forest Gridsearch\n",
      "Grid Search Best Score: 0.73375\n",
      "Grid Search Best Params: {'clf__n_estimators': 100, 'clf__bootstrap': True, 'clf__criterion': 'gini', 'clf__max_depth': 20, 'pca__n_components': 8, 'clf__min_samples_split': 4}\n",
      "Grid Search Best Estimator: Pipeline(steps=[('ss', StandardScaler(copy=True, with_mean=True, with_std=True)), ('pca', PCA(copy=True, iterated_power='auto', n_components=8, random_state=42,\n",
      "  svd_solver='auto', tol=0.0, whiten=False)), ('clf', RandomForestClassifier(bootstrap=True, class_weight=None, criterion='gini',\n",
      "            max_de...mators=100, n_jobs=1, oob_score=False, random_state=None,\n",
      "            verbose=0, warm_start=False))])\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.73      0.85      0.79       140\n",
      "          2       0.45      0.28      0.35        60\n",
      "\n",
      "avg / total       0.65      0.68      0.66       200\n",
      "\n",
      "Accuracy: 0.680\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHhVJREFUeJzt3XlUlPXiP/D3MOyLhAKikFwhxQVByEoSr6EoXzcEM82l\ntEwvKZqllcotzMysvFrpCRUVvOpVr4VclzCuqLjvoJErxISgF0FFAQVk5vP7w+P8mhDB5DMPy/t1\nDuf4LPM87xng7ef5zIJKCCFARCSBidIBiKjxYsEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgmlC5syZ\ng5YtW0KlUiE+Pv6Jj6fRaKBSqXDixIknD9cAvPTSS4iMjFQ6RoOi4utglJWfn4/58+dj+/btyM3N\nhaOjI3x8fDBlyhQMGDCgzs6TkZGBLl26ICEhAQEBAbC3t4eVldUTHVOr1aKgoACOjo4wNTWto6TG\no9Fo0LZtWxw/fhzdunWrcf8bN27AzMwMdnZ2RkjXODS8n4pGRKPRoEePHrCzs8Pnn38OX19f6HQ6\npKSkICIiAjk5OXV2rszMTABAWFgYVCpVnRxTrVbDxcWlTo5Vn1VUVMDc3BzNmzdXOkrDI0gx/fv3\nF61atRLFxcVVtt24cUP/799++02EhYUJW1tbYWtrK8LDw8Xly5f126Ojo0Xnzp3Fhg0bhIeHh7C1\ntRVDhgwRBQUF+u0ADL6EEGLs2LFi4MCBBud9cKwHzpw5I3r37i3s7OyEra2t8PHxEbt37xZCCJGd\nnS0AiOPHj+v3T01NFc8//7ywsLAQzs7OYtq0aaK8vFy/vVevXuLtt98Ws2bNEi1atBBOTk5i+vTp\nQqvVVvs4xcXFCRsbG/Hjjz8KLy8vYWVlJQYPHiyKiorE5s2bxTPPPCOaNWsmxowZI+7cuaO/XVJS\nkggMDBRPPfWUcHBwEP369RNnz57Vb//jY9KrVy+Dx2XBggXC1dVVODk56bNPnjxZCCHE+fPnhbW1\ntVizZo3B+czMzMThw4ervS9NDedgFHLjxg3s3LkTkZGRsLW1rbLdwcEBACCEQFhYGPLz87F7927s\n2bMHV65cQVhYGMTvrm41Gg02bdqELVu2IDk5GWlpaYiKigIAzJgxA7GxsQCAq1ev4urVq7XOOWrU\nKLRq1QrHjh1DWloa5syZA0tLy4fum5eXh/79+8PPzw9paWlYtWoVNmzYgFmzZhnst379epiamuLQ\noUNYunQpvv76a2zatOmROcrLy/GPf/wD69evR0pKCk6cOIFhw4ZhzZo1+OGHH5CYmIjt27fju+++\n09+mtLQU06ZNw7Fjx7B3717Y29tj8ODBqKioAAAcO3YMALBz505cvXoVCQkJ+tumpqbizJkz2Llz\nJ1JSUqrk8fLywuLFixEZGYmsrCwUFBRg3LhxiIqKQvfu3Wv34DYFSjdcU3X06FEBQCQkJDxyv+Tk\nZGFiYiKys7P167KysoRKpRL//e9/hRD3Rx0WFhaiqKhIv8+8efOEp6enfnnz5s3ij9/u2oxg7Ozs\nRHx8/EOz/XEEM3v2bOHp6WkwGomLixPm5uaitLRUCHF/FNC9e3eD4wQHB4vx48dX+xjExcUJAOL8\n+fP6ddOnTxcmJib6UVp19+f3SkpKhImJidi/f/9D8//+OI6OjqKsrMxg/e9HMA+EhYWJF154QQwY\nMEC8+OKLorKystrzN0UcwShE1HJu/dy5c2jdujX+8pe/6Nd5eHigdevWOHv2rH6du7s77O3t9cut\nW7fGtWvXnjjne++9h7feegu9e/fGZ599hvPnzz8ya0BAAExM/v+PVWBgICoqKvRzQADg4+NjcLva\nZLWwsICXl5d+uWXLlnBxcYGjo6PBut8fJysrC6NGjYKnpyeaNWuGli1bQqfT1Wpuy9vbGxYWFjXu\nt3LlSmRmZmLfvn1Yt24d1Gp1jbdpSlgwCmnXrh1UKhXOnTv3yP2EENVOyv5+vZmZWZVtOp3ukcc2\nMTGpUnT37t0zWJ4zZw7Onj2LsLAwHDp0CD4+Pli9erXRs/7xWSqVSlXjcQYPHoyCggIsX74cR48e\nRVpaGkxNTfWXSI9iY2NT4z7A/Wfnbt26hbKyMuTl5dXqNk0JC0YhzZs3R0hICJYuXYqSkpIq24uK\nigAAnTp1Ql5eHjQajX7br7/+iitXrqBTp05PlMHJyanKfEx6enqV/dq1a4epU6dix44dGD9+PFau\nXPnQ43Xq1AmHDx82+CU/cOAAzM3N4enp+URZH9f169dx7tw5zJ49G8HBwejYsSOKi4tRWVmp38fc\n3BzA/afb/4yioiK8/vrrmDFjBiZPnozXXnsNt2/frpP8jQULRkHfffcdhBDo1q0bNm/ejAsXLuD8\n+fOIiYnRX0YEBwfD19cXo0ePxsmTJ3HixAmMHj0a/v7+6N279xOdv3fv3khLS8Pq1auRmZmJL7/8\nEgcPHtRvv3v3LiZPnoy9e/dCo9Hg6NGjOHDgQLXFNmnSJFy5cgWTJk3CuXPnsGPHDsycORORkZGw\ntrZ+oqyPy8HBAY6OjoiNjUVmZiZSU1MRERFhMBJydnaGlZUVfvrpJ+Tn5+PWrVuPdY6IiAg4Ojpi\n7ty5+OKLL2BnZ4fJkyfX9V1p0FgwCmrbti1OnTqFvn374sMPP4SPjw969+6NrVu3Yvny5QDuD/sT\nExPh5OSEl156CUFBQXBxcUFiYuITv54lJCQE0dHRiIqKwrPPPguNRoNJkybpt6vVaty8eRNjx46F\nl5cXwsPDERAQgEWLFj30eK6urkhKSkJaWhq6du2KN998EyNHjsT8+fOfKOefYWJigk2bNuHMmTPw\n9vbG5MmT8emnnxrMq5iamuLbb7/FypUr0bp1awwZMqTWx1+7di22bt2K9evXw8zMDBYWFvjXv/6F\n77//Hhs2bJBxlxokvpKXiKThCIaIpGHBEJE0LBgikoYFQ0TS1It3U5eVlSEjIwNOTk58JSRRA/Pg\nYzu8vb2rvE+tXhRMRkYGRo8erXQMInoC69evr/K5OvWiYJycnADcD9gUPl+kKerQwVvpCCSJWm0C\nFxcn/e/x79WLgnlwWeTi4gI3NzeF05AMf/bl+NRwPGx6g5O8RCQNC4aIpGHBEJE0LBgikoYFQ0TS\nsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0L\nhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCI\nSBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0Lhoik\nYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgqkj+/btQ2hoKFxdXaFSqRAfH2+wPSEhASEhIXBycoJK\npcLevXurHCMrKwvh4eFwcnJCs2bNMHz4cOTn5xvnDtBjmTFjOg4c2Iv8/Dzk5GTj++//jU6dOhrs\nM2RIKLZu3YKcnGzcvVuMnj0DFUqrHBZMHSkpKYG3tze++eYbWFlZVdleWlqKF198EYsWLXro7UtL\nS9GvXz8IIZCSkoKDBw+ioqICgwcPhk6nkx2fHtNf/xqI5ctXIigoGP37D4RWW4kdO7bBwcFBv4+1\ntTWOHDmKDz+cpWBSZZkqHaCxGDBgAAYMGAAAGDduXJXtr732GgCgsLDwobc/ePAgsrOzceLECf0P\n6Zo1a+Dg4IDdu3cjODhYTnD6U0JDww2W33xzAvLz8xAQ0B0//pgEANiwYSMAoEWLFkbPV19wBFNP\nlJeXQ6VSwdLSUr/O0tISJiYmOHDggILJqDbs7GyhVqtRVFSkdJR6hQVTT3Tv3h22trZ4//33UVpa\nitLSUsyYMQNarRZXr15VOh7VYOHCL5GefhpHjhxVOkq9woKpJ5ycnLB582YkJSXBzs4O9vb2KCoq\ngr+/P9RqtdLx6BG++OJzvPhiAEaOHMP5sj+QOgezbt06JCQk4OLFixg0aBAWLFgg83QNXr9+/ZCV\nlYXCwkKYmpriqaeegouLC9q2bat0NKrGl19+jmHDhuH//m8gNBqN0nHqHakF4+zsjEmTJmH//v0o\nLy+XeapGxdHREQCwe/duXLt2DaGhoQonoodZuPALDBs2DCEhA3Dx4kWl49RLUgumX79+AICff/65\n0b+eo6SkBJmZmQAAnU6HnJwcpKeno3nz5mjTpg1u3LiBnJwc/SRgZmamfoTi4uICAIiLi0OHDh3g\n7OyMw4cP45133sG7774LLy8vxe4XPdzixf/AqFGvYvjwkSgquomWLZ0BACUl9+fPAMDBwQFPP+0G\ne3t7AICnpydu3bqF/Px85OdfUyy7MXEOpo6cOHECfn5+8PPzw927dxEdHQ0/Pz98/PHHAICtW7fC\nz88PQUFBAIAJEybAz88Py5Yt0x/jwoULCA8PR8eOHTF37lxERUVh4cKFitwferSIiIlo1qwZdu7c\nAY0mS/81bdpU/T4DBw7A0aOHkJx8/2nrmJilOHr0EN56a7xSsY1OJYQQsk+yePFi5OfnVzsHk5ub\niz59+iAlJQVubm6y45ACrKzslI5AkqjVari6tnzo7y9HMEQkDQuGiKSROslbWVkJrVYLnU4HrVaL\n8vJyqNVqmJryHQpETYHUEUxMTAx8fHywYsUKbN26FT4+PoiJiZF5SiKqR6QOJaZMmYIpU6bIPAUR\n1WOcgyEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCR\nNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnD\ngiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSmFa34fbt24+8YbNmzeo8DBE1LtUWTOfOnaFSqSCE\n0K97sKxSqZCTk2OUgETUcFVbMJcvXzZmDiJqhGo1B7Nx40bMnz8fAJCbm4uTJ09KDUVEjUONBRMZ\nGYk9e/Zg7dq1AABra2tERERID0ZEDV+1l0gPHDp0CKdOnYKfnx8AoHnz5qioqJAejIgavhpHMGZm\nZtDpdFCpVACA69evw8SEz24TUc1qbIrJkyfj5ZdfRkFBAaKjoxEYGIgPP/zQGNmIqIGr8RLp9ddf\nx7PPPotdu3YBADZv3gxvb2/pwYio4auxYABAq9XCzMwMKpUKOp1OdiYiaiRqvET67LPPMHLkSFy5\ncgW5ubkYNWoUPv/8c2NkI6IGrsYRzLp163Dy5ElYW1sDAKKiovDss89i1qxZ0sMRUcNW4wjG3d0d\nlZWV+uXKykp4eHhIDUVEjUO1I5h3330XKpUK1tbW6Ny5M0JCQqBSqZCcnIzAwEBjZiSiBqragnnw\nTFHnzp0xcOBA/fru3bvLT0VEjUK1BTN+/Hhj5iCiRqjGSd6srCxERUXh7NmzKCsr06+/ePGi1GBE\n1PDVOMk7btw4vPHGGxBCICkpCcOHD8err75qjGxE1MDVWDB37txBSEgIAMDT0xPz5s3Dnj17pAcj\nooavxkskCwsLCCHg6emJZcuWwdXVFdeuXTNGNiJq4GosmMWLF6OkpATffvstoqKicOvWLaxevdoY\n2YiogauxYF544QUAgJ2dnf5Dp4iIaqPaggkPD9d/BszDJCQkSAlERI1HtQUTGRlpzBwAgKFDJ0Kt\nNjf6eUm+srISpSOQJKam1V8IVbulT58+UsIQUdPBz74kImlYMEQkTa0Lpry8XGYOImqEaiyYY8eO\noUuXLmjXrh0A4PTp05gyZYr0YETU8NVYMFOnTsX27dvRokULAICvry/fKkBEtVJjweh0Ori7uxus\nU6vV0gIRUeNR4yt5n376aRw7dgwqlQparRZLlixB+/btjZGNiBq4GkcwMTExWLRoEXJyctCyZUsc\nOXIEMTExxshGRA1cjSMYZ2dnbNy40RhZiKiRqbFgJkyY8ND3JK1YsUJKICJqPGosmODgYP2/y8rK\nsGXLFjz99NNSQxFR41BjwYwYMcJg+bXXXkPfvn2lBSKixuOx3yqQnZ2N3377TUYWImpkahzBODg4\n6OdgdDodmjdvjgULFkgPRkQN3yMLRgiB06dPw9XVFQBgYmLyyA+hIiL6vUdeIqlUKoSHh0OtVkOt\nVrNciOix1DgH8/zzz+PUqVPGyEJEjUy1l0iVlZUwNTXFgQMHEBsbC09PT9jY2EAIAZVKxdIhohpV\nWzAPRi6JiYnGzENEjUi1BSOEAHD/rzkSEf0Z1RZMQUEBFi1aVO0N33vvPSmBiKjxqLZgtFotSkpK\n9CMZIqLHVW3BtGrVCh9//LExsxBRI1Pt09QcuRDRk6q2YFJSUoyZg4gaoWoLpnnz5sbMQUSNEP/w\nGhFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQ\nkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCMZOzYYTh8\nOBHTp0/Qr5s4cRQ2blyK3bs34qef1mHJkrno0sVLwZRUWz179sR//vMf5ObmQgiBsWPHGmwXQjz0\na+nSpQolVgYLxgg6d26P0NC+uHQp22D9b7/lYeHCFRgz5h1ERMzClSv5WLw4Gg4O9golpdqytbVF\nRkYG3nnnHdy5c6fKdhcXF4OvQYMGAQD+/e9/GzuqokyVDtDY2dhYY86c9zB//lK8+eYIg20//ZRq\nsPzNN6sRGtoX7du3xdGj6caMSY8pKSkJSUlJAID4+Pgq2/Pz8w2WhwwZggsXLmDfvn3GiFdvcAQj\n2cyZk7BnzyGcPPnzI/czNTVFWFg/lJSU4uLF7EfuSw2LjY0NXn31VcTGxiodxeg4gpEoNLQv3Nxa\n4ZNPvq52nx49umHu3OmwtLTA9es38c47c3Dz5i0jpiTZRo0aBQsLC6xZs0bpKEbHEYwkbdq0RkTE\nGMyZswiVlZXV7nfy5M8YO/ZdTJw4E0eOpGHevPfRooWDEZOSbBMmTEBiYiIKCwuVjmJ00gqmoqIC\ns2fPRlBQEPz8/BAWFobU1NSab9hIeHt3gIODPdat+xb79/+A/ft/gL+/N4YO7Y/9+3+Amdn9wWNZ\nWTlyc/+HX365iPnzl6KyshKhoX0VTk91xdfXF88991yTvDwCJF4iVVZWolWrVli7di1at26N1NRU\nTJs2Ddu2bYObm5us09Yb+/YdxejRUw3WRUVNQW7uVaxZ8z3u3Xv4qEalMtGXDzV8EydORHZ2Nnbt\n2qV0FEVI+0m2trbGlClT9MtBQUFwc3PDL7/80iQKpqSkFCUlpQbrysrKcft2MX79NQfW1lYYMyYc\nBw4cx/XrN/HUU/YYNqw/nJ1bICXloEKpqbZsbGzwzDPPAABMTEzQpk0b+Pr64saNG7h8+TIAwMrK\nCqNHj8aXX36pZFRFGe2/ysLCQmg0Gv03panTarXw8GiDQYOCYW9vh1u3inHu3CW8/fZsZGX9pnQ8\nqkG3bt2wd+9e/fLcuXMxd+5cxMfH44033gAAjBgxAjY2NoiLi1MopfJUQggh+yT37t3DhAkT0KZN\nG8ydO7fK9tzcXPTp0wf29p5Qq81lxyEFHDnyH6UjkCSmpqbw8PBASkpKlasT6c8i6XQ6fPDBBzAz\nM8NHH30k+3REVI9IvUQSQiAqKgqFhYWIjY2FmZmZzNMRUT0jtWCio6ORlZWFuLg4WFpayjwVEdVD\n0gomLy8PmzZtgrm5OQIDA/XrP/nkE4SGhso6LRHVI9IKxtXVFRcuXJB1eCJqAPhWASKShgVDRNKw\nYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuG\niKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhI\nGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRh\nwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYM\nEUljqnQAANBqtQAAne6ewklIFlPTevGjRhI8+N4++D022GbsMA9TUFAAACguzlE4Ccni4eGhdASS\nrKCgAO7u7gbrVEIIoVAevbKyMmRkZMDJyQlqtVrpOET0GLRaLQoKCuDt7Q1LS0uDbfWiYIioceIk\nLxFJw4IhImlYMEQkDQuGiKRhwRjZunXrMHToUHh7e2PmzJlKx6E6VFFRgdmzZyMoKAh+fn4ICwtD\namqq0rEUVS9eB9OUODs7Y9KkSdi/fz/Ky8uVjkN1qLKyEq1atcLatWvRunVrpKamYtq0adi2bRvc\n3NyUjqcIFoyR9evXDwDw888/Iz8/X+E0VJesra0xZcoU/XJQUBDc3Nzwyy+/NNmC4SUSkSSFhYXQ\naDR45plnlI6iGBYMkQT37t3DjBkzEB4eDk9PT6XjKIYFQ1THdDodPvjgA5iZmeGjjz5SOo6iOAdD\nVIeEEIiKikJhYSFiY2NhZmamdCRFsWCMrLKyElqtFjqdDlqtFuXl5VCr1fw4g0YiOjoaWVlZiIuL\nq/LGv6aIb3Y0siVLlmDp0qUG6yIjIw2efaCGKS8vD71794a5ubnBfxiffPIJQkNDFUymHBYMEUnD\nSV4ikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQumkVKr1ejatSu8vb3xyiuv4M6dO3/6WHv37sWg\nQYMAAFu3bsWCBQuq3beoqAjffffdY59jzpw5WLhwYa3X/964cePw/fff1/pcGo0G3t7ej52RHh8L\nppGysrJCeno6MjIyYG5ujmXLlhlsF0JAp9M99nFDQ0Mf+UFZf7ZgqHFiwTQBPXv2RGZmJjQaDTp2\n7IhJkybB398fly9fRnJyMgICAuDv749XXnkFJSUlAICdO3eiQ4cOCAwMREJCgv5Y8fHxiIyMBADk\n5+cjPDwcvr6+8PX1xaFDhzBz5kxkZWWha9eueP/99wEAX331FZ577jn4+PggOjpaf6zPPvsMXl5e\nCA4OxoULF2q8H7GxsXjuuefg6+uLl19+2WBUtmvXLvTs2RPt27fH9u3bAdz/ez3vv/++/tzLly9/\n8geTHgsLppGrrKxEUlISunTpAgC4cOECXn/9daSlpcHGxgbz5s3Drl27cOrUKXTr1g2LFi1CWVkZ\nJkyYgG3btmH//v343//+99BjT506Fb169cLp06dx6tQpdO7cGQsWLICnpyfS09Px1VdfITk5GZcu\nXcKxY8eQnp6OkydPYt++fTh58iQ2btyItLQ0JCQk4Pjx4zXel6FDh+L48eM4ffo0OnbsiFWrVum3\naTQapKamYseOHYiIiEBZWRlWrVoFe3t7HD9+HMePH0dsbCyys7Pr5oGlWuE77Bqpu3fvomvXrgDu\nj2DGjx+PK1euwN3dHd27dwcAHDlyBGfPnkWPHj0A3P9M2YCAAJw/fx5t27ZFu3btAABjxozBihUr\nqpxj9+7d+Oc//wng/pyPvb09bt68abBPcnIykpOT4efnBwAoKSnBpUuXUFxcjPDwcFhbWwNArd6r\nk5GRgb///e8oKipCSUkJQkJC9NuGDx8OExMTtGvXDh4eHjh//jySk5Nx5swZ/fzMrVu3cOnSJbRv\n3772DyQ9ERZMI/VgDuaPbGxs9P8WQqBv377YsGGDwT7p6elQqVR1kkMIgVmzZuFvf/ubwfqvv/76\nsc8xbtw4JCYmwtfXF/Hx8di7d69+2x+PpVKpIITAkiVLDIoIuD/aIePgJVIT1r17dxw8eBCZmZkA\ngDt37uDixYvo0KEDsrOzkZWVBQBVCuiBPn36ICYmBsD9+Y7bt2/Dzs4OxcXF+n1CQkKwevVq/dxO\nXl4erl27hr/+9a/YsmUL7t69i+LiYmzbtq3GvMXFxWjVqhXu3buH9evXG2zbvHkzdDodsrKy8Ouv\nv8LLywshISGIiYnBvXv3AAAXL15EaWnpYz5K9CQ4gmnCnJycEB8fj5EjR+r/wsG8efPQvn17rFix\nAgMHDoSjoyMCAwORkZFR5fbffPMNJk6ciFWrVkGtViMmJgYBAQHo0aMHvL290b9/f3z11Vc4d+4c\nAgICAAC2trZYt24d/P39MWLECHTt2hXu7u7o2bNnjXk//fRTvPDCC3B3d0eXLl0MiszLywu9evVC\nfn4+li1bBktLS7z11lvQaDTw9/eHEAJOTk5ITEyso0ePaoMf10BE0vASiYikYcEQkTQsGCKShgVD\nRNKwYIhIGhYMEUnDgiEiaVgwRCTN/wM7dSYl7cw1qwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1255cad10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pipeline = Pipeline([\n",
    "        ('ss', StandardScaler()),\n",
    "        ('pca', PCA(random_state=42)),\n",
    "        ('clf', rf)\n",
    "    ])\n",
    "grid_search = GridSearchCV(pipeline, rf_parameters, verbose=1, cv=5)\n",
    "grid_search.fit(data_train, label_train)\n",
    "print \"Ranfom Forest Gridsearch\"\n",
    "print \"Grid Search Best Score:\", grid_search.best_score_\n",
    "print \"Grid Search Best Params:\", grid_search.best_params_\n",
    "print \"Grid Search Best Estimator:\", grid_search.best_estimator_\n",
    "    \n",
    "predictions = grid_search.best_estimator_.predict(data_test)\n",
    "print \"Classification Report:\", classification_report(label_test, predictions)\n",
    "print 'Accuracy: {0:.3f}'.format(accuracy_score(label_test, predictions))  \n",
    "\n",
    "    \n",
    "cnf_matrix = confusion_matrix(label_test, predictions)\n",
    "plot_confusion_matrix(cnf_matrix, classes=grid_search.best_estimator_.classes_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best score generated by the model is 0.73375 compared to 0.72625 on the non hyper parameter tuned Random Forest. This minor improvement is good but once again the Random Forest underperforms when measuring the accuracy on the test hold out data with an accuracy of only 0.68, this means that there is overfitting to the training data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gridsearching Logistic Regression w/ PCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lr = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lr_parameters = {\n",
    "           \"clf__penalty\"      : ['l2'],\n",
    "           # PENALTY SET WITH RIDGE DUE TO OTHER SOLVERS ONLY USING RIDGE\n",
    "    \n",
    "           \"clf__C\"            : [0.001, 0.01, 0.1, 1, 100],\n",
    "           # INVERSE OF REGULARIZATION STRENGTH\n",
    "    \n",
    "           \"clf__solver\"       : ['liblinear', 'sag'],\n",
    "           # ALGORITHM USED IN OPTIMIZATION PROBLEM\n",
    "    \n",
    "           \"clf__tol\"          : [0.0001, 0.001, 0.01],\n",
    "           # TOLERANCE FOR STOPPING CRITERIA\n",
    "    \n",
    "           \"pca__n_components\" : [2, 8, 10, 20, 25, 29]\n",
    "           # NO. OF PRINCIPAL COMPONENTS\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 120 candidates, totalling 600 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done 600 out of 600 | elapsed:    9.2s finished\n",
      "//anaconda/lib/python2.7/site-packages/matplotlib/font_manager.py:1297: UserWarning: findfont: Font family [u'sansserif'] not found. Falling back to DejaVu Sans\n",
      "  (prop.get_family(), self.defaultFamily[fontext]))\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logistic Regression Gridsearch\n",
      "Grid Search Best Score: 0.7325\n",
      "Grid Search Best Params: {'clf__tol': 0.0001, 'clf__penalty': 'l2', 'clf__C': 0.01, 'clf__solver': 'liblinear', 'pca__n_components': 10}\n",
      "Grid Search Best Estimator: Pipeline(steps=[('ss', StandardScaler(copy=True, with_mean=True, with_std=True)), ('pca', PCA(copy=True, iterated_power='auto', n_components=10, random_state=42,\n",
      "  svd_solver='auto', tol=0.0, whiten=False)), ('clf', LogisticRegression(C=0.01, class_weight=None, dual=False, fit_intercept=True,\n",
      "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
      "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
      "          verbose=0, warm_start=False))])\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.78      0.86      0.82       140\n",
      "          2       0.57      0.42      0.48        60\n",
      "\n",
      "avg / total       0.71      0.73      0.72       200\n",
      "\n",
      "Accuracy: 0.730\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHrNJREFUeJzt3XlYVPXiP/D3MCzKIoGAgiQpIqkIYpZyxWuoyTWXQDOv\nS+nN7JqCmZobt4uVW5tW+guTFLrqNR+7Ri5pJCpqlpKAhIoKMbIoW4ayyDLM5/eHXycnRDT4zJHx\n/Xoenoc5nzPnvGeQt59zzsygEkIIEBFJYKZ0ACIyXSwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDAP\nkCVLlqBdu3ZQqVSIjY1t8vY0Gg1UKhV++umnpodrAZ588kmEhYUpHaNFUfF1MMoqLCzE8uXLsXv3\nbuTl5cHJyQm+vr4IDw/H008/3Wz7SU9PR8+ePbFjxw4EBATA3t4erVu3btI26+rqUFxcDCcnJ5ib\nmzdTUuPRaDTo1KkTkpKS0KdPn0bXv3LlCiwsLGBnZ2eEdKah5f2rMCEajQb9+/eHnZ0dVqxYAT8/\nP+h0OiQkJGD69OnIyclptn1lZmYCAEJCQqBSqZplm2q1Gu3bt2+Wbd3PampqYGlpCUdHR6WjtDyC\nFDNs2DDh6uoqysrK6o1duXJF//3FixdFSEiIsLW1Fba2tiI0NFTk5ubqxyMjI0WPHj3E1q1bRefO\nnYWtra145plnRHFxsX4cgMGXEEJMnjxZDB8+3GC/N7d1U1pamhg0aJCws7MTtra2wtfXVxw4cEAI\nIUR2drYAIJKSkvTrJyYmiieeeEJYWVkJFxcXMXv2bFFdXa0fHzhwoHjllVfEokWLRNu2bYWzs7OY\nO3euqKura/B5iomJETY2NuKbb74R3t7eonXr1mLkyJGitLRUbN++XXTp0kW0adNGTJo0SVRWVurv\nt3fvXhEYGCgeeugh4eDgIIYOHSrOnDmjH//jczJw4ECD52XlypWiQ4cOwtnZWZ995syZQgghMjIy\nhLW1tfj8888N9mdhYSF++OGHBh/Lg4bnYBRy5coV7Nu3D2FhYbC1ta037uDgAAAQQiAkJASFhYU4\ncOAADh48iEuXLiEkJATilqNbjUaDbdu24auvvkJ8fDxSUlIQEREBAJg3bx6io6MBAJcvX8bly5fv\nOueECRPg6uqKEydOICUlBUuWLEGrVq1uu25+fj6GDRsGf39/pKSkYMOGDdi6dSsWLVpksN6WLVtg\nbm6OY8eOYe3atfjwww+xbdu2O+aorq7GBx98gC1btiAhIQE//fQTnn32WXz++ef43//+h7i4OOze\nvRuffPKJ/j4VFRWYPXs2Tpw4gUOHDsHe3h4jR45ETU0NAODEiRMAgH379uHy5cvYsWOH/r6JiYlI\nS0vDvn37kJCQUC+Pt7c3Vq9ejbCwMGRlZaG4uBhTpkxBREQE+vXrd3dP7oNA6YZ7UB0/flwAEDt2\n7LjjevHx8cLMzExkZ2frl2VlZQmVSiW+++47IcSNWYeVlZUoLS3Vr7N06VLh6empv719+3bxxx/3\n3cxg7OzsRGxs7G2z/XEGs3jxYuHp6WkwG4mJiRGWlpaioqJCCHFjFtCvXz+D7QwZMkRMnTq1wecg\nJiZGABAZGRn6ZXPnzhVmZmb6WVpDj+dW5eXlwszMTBw5cuS2+W/djpOTk6iqqjJYfusM5qaQkBDR\nt29f8fTTT4u//OUvQqvVNrj/BxFnMAoRd3lu/ezZs3Bzc8MjjzyiX9a5c2e4ubnhzJkz+mUeHh6w\nt7fX33Zzc0NRUVGTc86ZMwcvvfQSBg0ahGXLliEjI+OOWQMCAmBm9vs/q8DAQNTU1OjPAQGAr6+v\nwf3uJquVlRW8vb31t9u1a4f27dvDycnJYNmt28nKysKECRPg6emJNm3aoF27dtDpdHd1bsvHxwdW\nVlaNrvfZZ58hMzMThw8fxubNm6FWqxu9z4OEBaMQLy8vqFQqnD179o7rCSEaPCl763ILC4t6Yzqd\n7o7bNjMzq1d0tbW1BreXLFmCM2fOICQkBMeOHYOvry82btxo9Kx/vEqlUqka3c7IkSNRXFyMTz/9\nFMePH0dKSgrMzc31h0h3YmNj0+g6wI2rc1evXkVVVRXy8/Pv6j4PEhaMQhwdHREcHIy1a9eivLy8\n3nhpaSkAoHv37sjPz4dGo9GP/fLLL7h06RK6d+/epAzOzs71zsekpqbWW8/LywuzZs3Cnj17MHXq\nVHz22We33V737t3xww8/GPySHz16FJaWlvD09GxS1nv166+/4uzZs1i8eDGGDBmCbt26oaysDFqt\nVr+OpaUlgBuX2/+M0tJSvPDCC5g3bx5mzpyJ559/HteuXWuW/KaCBaOgTz75BEII9OnTB9u3b8e5\nc+eQkZGBqKgo/WHEkCFD4Ofnh4kTJ+LkyZP46aefMHHiRPTu3RuDBg1q0v4HDRqElJQUbNy4EZmZ\nmXj33Xfx/fff68evX7+OmTNn4tChQ9BoNDh+/DiOHj3aYLHNmDEDly5dwowZM3D27Fns2bMHCxcu\nRFhYGKytrZuU9V45ODjAyckJ0dHRyMzMRGJiIqZPn24wE3JxcUHr1q3x7bfforCwEFevXr2nfUyf\nPh1OTk5466238M4778DOzg4zZ85s7ofSorFgFNSpUyckJyfjqaeewoIFC+Dr64tBgwZh586d+PTT\nTwHcmPbHxcXB2dkZTz75JIKCgtC+fXvExcU1+fUswcHBiIyMREREBB577DFoNBrMmDFDP65Wq/Hb\nb79h8uTJ8Pb2RmhoKAICArBq1arbbq9Dhw7Yu3cvUlJS0KtXL7z44osYP348li9f3qScf4aZmRm2\nbduGtLQ0+Pj4YObMmXj77bcNzquYm5vj448/xmeffQY3Nzc888wzd739TZs2YefOndiyZQssLCxg\nZWWF//73v/jyyy+xdetWGQ+pReIreYlIGs5giEgaFgwRScOCISJpWDBEJM198W7qqqoqpKenw9nZ\nma+EJGphbn5sh4+PT733qd0XBZOeno6JEycqHYOImmDLli31PlfnvigYZ2dnADcCPgifL/Ig6tSp\nk9IRSBJzc3N07NhR/3tsMKZAnnpuHha1b98e7u7uCqchGW59iT6Zptud3uBJXiKShgVDRNKwYIhI\nGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRh\nwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYM\nEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCR\nNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcE0k8OHD2PUqFHo0KEDVCoVYmNj9WO1\ntbVYsGABfH19YWNjA1dXV0yYMAE5OTkG21i/fj2CgoLw0EMPQaVSQaPRGPdB0D0ZMGAAvv76a+Tl\n5UEIgcmTJxuMu7i4ICYmBvn5+aioqMDevXvRpUsXhdIqgwXTTMrLy+Hj44OPPvoIrVu3NhirrKxE\ncnIyIiIikJycjK+//hq5ubn429/+Bq1Wa7De0KFDsWTJEiOnpz/D1tYW6enpePXVV1FZWVlvPC4u\nDl5eXggJCYG/vz8uXryI/fv3w9raWoG0ChH3gdzcXNG1a1eRm5urdJRmYWNjI2JiYu64zunTpwUA\nkZaWVm8sKSlJABDZ2dlyAioAgEl/lZWVicmTJ+tve3l5CSGE8PX11S9TqVSisLBQTJ06VfG8zfll\nbm7e4O8vZzAKuXbtGgDAwcFB4SQkg5WVFQCgqqpKv0wIgerqagQGBioVy+hYMAqoqanB3LlzMXLk\nSLi7uysdhyTIyMiARqPB8uXL4eDgAAsLC8yfPx8PP/wwXF1dlY5nNCwYI9NqtZg0aRJKS0sRExOj\ndBySRKvVYsyYMfD09MSVK1dQWVmJoKAgfPPNN6irq1M6ntFILZjNmzdj9OjR8PHxwcKFC2XuqkXQ\narUYP3480tLSkJCQgLZt2yodiSRKTk6Gv78/7O3t4erqimHDhqFt27bIzs5WOprRSC0YFxcXzJgx\nA2PGjJG5mxahtrYW48aNQ1paGg4ePIj27dsrHYmM5Nq1aygpKUGXLl3Qp08ffP3110pHMhpzmRsf\nOnQoAODnn39GYWGhzF0prry8HJmZmQAAnU6HnJwcpKamwtHREW5ubhg7diySkpKwa9cuqFQqFBQU\nAADs7e31l7ULCgpQUFCA8+fPAwDOnDmD0tJSdOzYEY6Ojso8MGqQjY2N/nUtZmZm6NixI/z8/HDl\nyhXk5ubi2WefRUlJCS5evIiePXvio48+QlxcHL777juFkxuRMS5Rrlq1SixYsKDBcVO4TH3w4MHb\nXsKbPHmyyM7ObvAS362XsyMjIxtdp6Vq6PG35K+BAwfe9rHGxMQIACI8PFzk5OSI6upqodFoxFtv\nvSUsLCwUz93cX3e6TK36vx++VKtXr0ZhYSFWrlx52/G8vDwMHjwYCQkJvKpiolQqldIRSBJzc3N0\n7tz5tr+/vIpERNKwYIhIGqknebVaLerq6qDT6VBXV4fq6mqo1WqYm0vdLRHdJ6TOYKKiouDr64v1\n69dj586d8PX1RVRUlMxdEtF9ROpUIjw8HOHh4TJ3QUT3MZ6DISJpWDBEJA0LhoikYcEQkTQsGCKS\nhgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVD\nRNKYNzRw7dq1O96xTZs2zR6GiExLgwXTo0cPqFQqCCH0y27eVqlUyMnJMUpAImq5GiyY3NxcY+Yg\nIhN0V+dgvvjiCyxfvhwAkJeXh5MnT0oNRUSmodGCCQsLw8GDB7Fp0yYAgLW1NaZPny49GBG1fA0e\nIt107NgxJCcnw9/fHwDg6OiImpoa6cGIqOVrdAZjYWEBnU4HlUoFAPj1119hZsar20TUuEabYubM\nmRgzZgyKi4sRGRmJwMBALFiwwBjZiKiFa/QQ6YUXXsBjjz2G/fv3AwC2b98OHx8f6cGIqOVrtGAA\noK6uDhYWFlCpVNDpdLIzEZGJaPQQadmyZRg/fjwuXbqEvLw8TJgwAStWrDBGNiJq4RqdwWzevBkn\nT56EtbU1ACAiIgKPPfYYFi1aJD0cEbVsjc5gPDw8oNVq9be1Wi06d+4sNRQRmYYGZzCvvfYaVCoV\nrK2t0aNHDwQHB0OlUiE+Ph6BgYHGzEhELVSDBXPzSlGPHj0wfPhw/fJ+/frJT0VEJqHBgpk6daox\ncxCRCWr0JG9WVhYiIiJw5swZVFVV6ZefP39eajAiavkaPck7ZcoU/OMf/4AQAnv37sVzzz2Hv//9\n78bIRkQtXKMFU1lZieDgYACAp6cnli5dioMHD0oPRkQtX6OHSFZWVhBCwNPTE+vWrUOHDh1QVFRk\njGxE1MI1WjCrV69GeXk5Pv74Y0RERODq1avYuHGjMbIRUQvXaMH07dsXAGBnZ6f/0CkiorvRYMGE\nhobqPwPmdnbs2CElEBGZjgYLJiwszJg5AABPDhwO/qkm02Rn56h0BJJErW74d7bBghk8eLCUMET0\n4OB0gYikYcEQkTR3XTDV1dUycxCRCWq0YE6cOIGePXvCy8sLAHDq1CmEh4dLD0ZELV+jBTNr1izs\n3r0bbdu2BQD4+fnxrQJEdFcaLRidTgcPDw+DZWq1WlogIjIdjb6S9+GHH8aJEyegUqlQV1eHNWvW\noGvXrsbIRkQtXKMzmKioKKxatQo5OTlo164dfvzxR0RFRRkjGxG1cI3OYFxcXPDFF18YIwsRmZhG\nC2batGm3fU/S+vXrpQQiItPRaMEMGTJE/31VVRW++uorPPzww1JDEZFpaLRgxo0bZ3D7+eefx1NP\nPSUtEBGZjnt+q0B2djYuXrwoIwsRmZhGZzAODg76czA6nQ6Ojo5YuXKl9GBE1PLdsWCEEDh16hQ6\ndOgAADAzM7vjh1AREd3qjodIKpUKoaGhUKvVUKvVLBciuieNnoN54oknkJycbIwsRGRiGjxE0mq1\nMDc3x9GjRxEdHQ1PT0/Y2NhACAGVSsXSIaJGNVgwN2cucXFxxsxDRCakwYIRQgC48dcciYj+jAYL\npri4GKtWrWrwjnPmzJESiIhMR4MFU1dXh/Lycv1MhojoXjVYMK6urvj3v/9tzCxEZGIavEzNmQsR\nNVWDBZOQkGDMHERkghosGEdH/qlPImoa/uE1IpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0Lhoik\nYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoW\nDBFJw4IhImlYMEQkDQuGiKRhwUgyadJz2L1nG1JTjyA19Qi2b/8cTz4ZqB9/5903kZmVYvD15Zef\nK5iY7sWcObNx6NB+5OVp8Msv57Bt2xZ06/aowTpRUWtx7dqvBl8JCd8qlFgZDf5tamqagoIivPvu\nx7ioyYFKpcLoMSMRtW4VQp6ZiHPnLgAAjh79EfPm/kt/n9raWqXi0j0aMKA/oqM3Ijk5GSqVChER\ni7Bz5w488cRf8Ntvpfr1Dhw4hJdffkV/u7a2Rom4imHBSLJ//yGD26s++H+YMGEs/Hv76gumpqYG\nJSW/KpCOmio0dKzB7ZdffgV5edno27cv9u37fZZSU1ONoqIiY8e7b/AQyQjMzMwwfEQwrK2tkXzy\nlH55nz7+OH4iAd/tj8Oy5W/Asa2DgimpKWxtbaFWq1FaWmqwvF+/fsjKykBy8nF8/PFqODk5KZRQ\nGSohhFA6RF5eHgYPHow6bSuYUud17doF27/8HFZWlqisvI45ry3GoUNHAQDDRwSj6noVcnPz4e7u\nhtfmzIRabYaQZyagpsb0DpWKinOUjiBVbOwGeHp2xsCBg6HT6QAAY8aEorLyOi5evIiOHTvijTcW\nQ61W469/HYSaGtM5VFKrzeDi4oiEhAS4u7sbjLFgJLKwMIebmyvs2tjhb8GD8dy4UEycOA0XzmfV\nW9fFxRmJh/fg1VkLER9/QIG0cplywSxf/jbGjBmN4OCnodFcbHC99u3b4/TpVEyZ8hJ27dptxIRy\n3algpP0219TUYPHixQgKCoK/vz9CQkKQmJgoa3f3pdpaLS5ezEX6z2fw/vtrcPbsebz4j0m3Xbeo\nqBgFBUV45JGORk5JTbFixVI8++xojBgRcsdyAYCCggLk51+Cp2dnI6VTnrSC0Wq1cHV1xaZNm3Dy\n5Em8+uqrmD17NvLy8mTt8r5nZqaCpaXFbcccHB5Cu3YuKCouMXIq+rPeeWc5xo4dgxEjQnHhwoVG\n13d0dISbmysKCwuNkO7+IO0qkrW1NcLDw/W3g4KC4O7ujtOnT9ebRpmi11+fhYMHj+Dy5QLY2Nhg\n1Khh6Nu3D156aRasrVtj1qvT8e2+BBQVFcPd3Q3z5s3ClV+v4DsTPDwyRR988C7GjXsOEyY8j9LS\nUri4uAAAKioqUFFRARsbGyxaNB87d+5CQUEhOnbsiCVL3kBxcQl27dqjcHrjMdpl6pKSEmg0GnTp\n0sVYu1SUk3NbfLBqGZyd2qKsvBwZGRcw9cUwHDnyA6ysrODdtQtCQ0fAzs4OxcUl+PHHJMyaNR8V\nFZVKR6e7MG3aVADA7t1xBstXrHgHK1a8i7q6OvTo0R3jx4+Dvb09CgoKceTIUUye/CLKy8uViKwI\no5zkra2txbRp09CxY0e89dZb9cZN9SQv/c6UT/I+6BQ5yXuTTqfD/PnzYWFhgTfeeEP27ojoPiL1\nEEkIgYiICJSUlCA6OhoWFrc/wUlEpklqwURGRiIrKwsxMTFo1aqVzF0R0X1IWsHk5+dj27ZtsLS0\nRGDg7+8ifvPNNzFq1ChZuyWi+4i0gunQoQPOnTsna/NE1ALwkg0RScOCISJpWDBEJA0LhoikYcEQ\nkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJ\nw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQs\nGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4Ih\nImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikMVc6AADU1dX9\n33c6RXOQPGo1/y8zVWZmN362v/8e/+6+KJji4mIAgNq8RuEkJIuLi6PSEUiy4uJieHh4GCxTCSGE\nQnn0qqqqkJ6eDmdnZ6jVaqXjENE9qKurQ3FxMXx8fNCqVSuDsfuiYIjINPHAmIikYcEQkTQsGCKS\nhgVDRNKwYIxs8+bNGD16NHx8fLBw4UKl41AzqqmpweLFixEUFAR/f3+EhIQgMTFR6ViKui9eB/Mg\ncXFxwYwZM3DkyBFUV1crHYeakVarhaurKzZt2gQ3NzckJiZi9uzZ2LVrF9zd3ZWOpwgWjJENHToU\nAPDzzz+jsLBQ4TTUnKytrREeHq6/HRQUBHd3d5w+ffqBLRgeIhFJUlJSAo1Ggy5duigdRTEsGCIJ\namtrMW/ePISGhsLT01PpOIphwRA1M51Oh/nz58PCwgJvvPGG0nEUxXMwRM1ICIGIiAiUlJQgOjoa\nFhYWSkdSFAvGyLRaLerq6qDT6VBXV4fq6mqo1WqYm/NHYQoiIyORlZWFmJiYem/8exDxzY5GtmbN\nGqxdu9ZgWVhYmMHVB2qZ8vPzMWjQIFhaWhr8h/Hmm29i1KhRCiZTDguGiKThSV4ikoYFQ0TSsGCI\nSBoWDBFJw4IhImlYMEQkDQvGRKnVavTq1Qs+Pj4YO3YsKisr//S2Dh06hBEjRgAAdu7ciZUrVza4\nbmlpKT755JN73seSJUvw/vvv3/XyW02ZMgVffvnlXe9Lo9HAx8fnnjPSvWPBmKjWrVsjNTUV6enp\nsLS0xLp16wzGhRDQ6e79D92NGjXqjh+U9WcLhkwTC+YBMGDAAGRmZkKj0aBbt26YMWMGevfujdzc\nXMTHxyMgIAC9e/fG2LFjUV5eDgDYt28fHn30UQQGBmLHjh36bcXGxiIsLAwAUFhYiNDQUPj5+cHP\nzw/Hjh3DwoULkZWVhV69euH1118HALz33nt4/PHH4evri8jISP22li1bBm9vbwwZMgTnzp1r9HFE\nR0fj8ccfh5+fH8aMGWMwK9u/fz8GDBiArl27Yvfu3QBu/L2e119/Xb/vTz/9tOlPJt0TFoyJ02q1\n2Lt3L3r27AkAOHfuHF544QWkpKTAxsYGS5cuxf79+5GcnIw+ffpg1apVqKqqwrRp07Br1y4cOXIE\nBQUFt932rFmzMHDgQJw6dQrJycno0aMHVq5cCU9PT6SmpuK9995DfHw8Lly4gBMnTiA1NRUnT57E\n4cOHcfLkSXzxxRdISUnBjh07kJSU1OhjGT16NJKSknDq1Cl069YNGzZs0I9pNBokJiZiz549mD59\nOqqqqrBhwwbY29sjKSkJSUlJiI6ORnZ2dvM8sXRX+A47E3X9+nX06tULwI0ZzNSpU3Hp0iV4eHig\nX79+AIAff/wRZ86cQf/+/QHc+EzZgIAAZGRkoFOnTvDy8gIATJo0CevXr6+3jwMHDuA///kPgBvn\nfOzt7fHbb78ZrBMfH4/4+Hj4+/sDAMrLy3HhwgWUlZUhNDQU1tbWAHBX79VJT0/Hv/71L5SWlqK8\nvBzBwcH6seeeew5mZmbw8vJC586dkZGRgfj4eKSlpenPz1y9ehUXLlxA165d7/6JpCZhwZiom+dg\n/sjGxkb/vRACTz31FLZu3WqwTmpqKlQqVbPkEEJg0aJF+Oc//2mw/MMPP7znfUyZMgVxcXHw8/ND\nbGwsDh06pB/747ZUKhWEEFizZo1BEQE3ZjtkHDxEeoD169cP33//PTIzMwEAlZWVOH/+PB599FFk\nZ2cjKysLAOoV0E2DBw9GVFQUgBvnO65duwY7OzuUlZXp1wkODsbGjRv153by8/NRVFSEv/71r/jq\nq69w/fp1lJWVYdeuXY3mLSsrg6urK2pra7FlyxaDse3bt0On0yErKwu//PILvL29ERwcjKioKNTW\n1gIAzp8/j4qKint8lqgpOIN5gDk7OyM2Nhbjx4/X/4WDpUuXomvXrli/fj2GDx8OJycnBAYGIj09\nvd79P/roI7z88svYsGED1Go1oqKiEBAQgP79+8PHxwfDhg3De++9h7NnzyIgIAAAYGtri82bN6N3\n794YN24cevXqBQ8PDwwYMKDRvG+//Tb69u0LDw8P9OzZ06DIvL29MXDgQBQWFmLdunVo1aoVXnrp\nJWg0GvTu3RtCCDg7OyMuLq6Znj26G/y4BiKShodIRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoW\nDBFJw4IhImn+P9LsD2wgWui9AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10272e390>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pipeline = Pipeline([\n",
    "        ('ss', StandardScaler()),\n",
    "        ('pca', PCA(random_state=42)),\n",
    "        ('clf', lr)\n",
    "    ])\n",
    "grid_search = GridSearchCV(pipeline, lr_parameters, verbose=1, cv=5)\n",
    "grid_search.fit(data_train, label_train)\n",
    "print \"Logistic Regression Gridsearch\"\n",
    "print \"Grid Search Best Score:\", grid_search.best_score_\n",
    "print \"Grid Search Best Params:\", grid_search.best_params_\n",
    "print \"Grid Search Best Estimator:\", grid_search.best_estimator_\n",
    "    \n",
    "predictions = grid_search.best_estimator_.predict(data_test)\n",
    "print \"Classification Report:\", classification_report(label_test, predictions)\n",
    "print 'Accuracy: {0:.3f}'.format(accuracy_score(label_test, predictions))  \n",
    "    \n",
    "cnf_matrix = confusion_matrix(label_test, predictions)\n",
    "plot_confusion_matrix(cnf_matrix, classes=grid_search.best_estimator_.classes_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The score improved very slightly on logistic regression from 0.7315 to 0.7325 on the grid search but however performs a bit better on classification accuracy on the test hold out data, at 0.73 rather than 0.715. The translation from the training data to test data actually improves which shows it is likely more reliable with new data than the Random Forest."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Gridsearching Support Vector Classifier w/ PCA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "svc = SVC(random_state=42)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "svc_parameters = {\n",
    "           \"clf__kernel\"      : ['rbf', 'linear'],\n",
    "           # KERNEL TYPE TO BE USED\n",
    "    \n",
    "           \"clf__C\"           : [0.1, 1, 10, 100],\n",
    "           # PENALTY PARAMETER FOR ERROR TERM\n",
    "    \n",
    "           \"clf__gamma\"       : [0.001, 0.01, 0.1, 1],\n",
    "           # KERNAL COEFFICIENT (SMOOTHING)\n",
    "    \n",
    "           \"pca__n_components\": [8, 10, 20, 25, 29]\n",
    "           # NO. OF PRINCIPAL COMPONENTS\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 480 candidates, totalling 2400 fits\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done 2400 out of 2400 | elapsed:  8.7min finished\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Logistic Regression Gridsearch\n",
      "Grid Search Best Score: 0.73125\n",
      "Grid Search Best Params: {'clf__gamma': 0.001, 'clf__degree': 1, 'pca__n_components': 20, 'clf__C': 100, 'clf__kernel': 'rbf'}\n",
      "Grid Search Best Estimator: Pipeline(steps=[('ss', StandardScaler(copy=True, with_mean=True, with_std=True)), ('pca', PCA(copy=True, iterated_power='auto', n_components=20, random_state=42,\n",
      "  svd_solver='auto', tol=0.0, whiten=False)), ('clf', SVC(C=100, cache_size=200, class_weight=None, coef0=0.0,\n",
      "  decision_function_shape=None, degree=1, gamma=0.001, kernel='rbf',\n",
      "  max_iter=-1, probability=False, random_state=42, shrinking=True,\n",
      "  tol=0.001, verbose=False))])\n",
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.79      0.85      0.82       148\n",
      "          2       0.45      0.35      0.39        52\n",
      "\n",
      "avg / total       0.70      0.72      0.71       200\n",
      "\n",
      "Accuracy: 0.720\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAH0ZJREFUeJzt3XlYVPXiP/D3MCzKIlcClaUoSUlBFrUUBRU3NNMgL6bm\nbt68Cl4r17iFmWt6zdInVFJQ8KrXNHMJokhQcwEFRFJUkEHcEDVUJJZhPr8//DlfCRFMPjMC79fz\n+DzM+Zw55z0DvP2cM2cGhRBCgIhIAgN9ByCihosFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQumEZk3\nbx5atmwJhUKByMjIp96eSqWCQqHA8ePHnz5cPdCrVy8EBQXpO0a9ouB1MPqVn5+PRYsWYe/evbh0\n6RKsra3h5uaG4OBgvP7663W2n4yMDHTo0AE7d+6El5cXLC0t0bRp06faZkVFBQoKCmBtbQ1DQ8M6\nSqo7KpUKL730EpKTk9G5c+ca17916xaMjIxgYWGhg3QNQ/37qWhAVCoVunfvDgsLCyxevBju7u7Q\naDSIj4/H5MmTcfHixTrbV1ZWFgDA398fCoWiTrapVCrRqlWrOtnWs6ysrAzGxsawsrLSd5T6R5De\nDBw4UNja2oq7d+9WGbt165b269zcXOHv7y/Mzc2Fubm5CAgIEHl5edrx0NBQ4eLiIrZs2SJat24t\nzM3NxZtvvikKCgq04wAq/RNCiLFjx4pBgwZV2u+DbT2Qnp4uevfuLSwsLIS5ublwc3MTv/zyixBC\niJycHAFAJCcna9dPTEwUr732mjAxMREtWrQQ06dPF6Wlpdrxnj17in/+859i7ty54rnnnhM2Njbi\nww8/FBUVFdU+TxEREcLMzEz88MMPwtnZWTRt2lQMHjxYFBYWiu3bt4uXX35ZNGvWTIwaNUoUFxdr\n7xcTEyO8vb3F3/72N9G8eXPRv39/cfr0ae34n5+Tnj17VnpelixZIuzt7YWNjY02+9SpU4UQQmRm\nZgpTU1OxcePGSvszMjISR44cqfaxNDY8B6Mnt27dQmxsLIKCgmBubl5lvHnz5gAAIQT8/f2Rn5+P\nX375Bfv378eVK1fg7+8P8dDRrUqlwrZt2/Ddd98hLi4OqampCAkJAQDMmDED4eHhAICrV6/i6tWr\ntc45cuRI2NraIikpCampqZg3bx6aNGnyyHUvX76MgQMHwtPTE6mpqVi/fj22bNmCuXPnVlpv8+bN\nMDQ0xOHDh7F69WqsXLkS27Zte2yO0tJS/Oc//8HmzZsRHx+P48eP4+9//zs2btyIHTt2YNeuXdi7\ndy++/vpr7X3u3buH6dOnIykpCQkJCbC0tMTgwYNRVlYGAEhKSgIAxMbG4urVq9i5c6f2vomJiUhP\nT0dsbCzi4+Or5HF2dsYXX3yBoKAgZGdno6CgAOPGjUNISAi6du1auye3MdB3wzVWx44dEwDEzp07\nH7teXFycMDAwEDk5Odpl2dnZQqFQiJ9++kkIcX/WYWJiIgoLC7XrLFiwQDg5OWlvb9++Xfz5212b\nGYyFhYWIjIx8ZLY/z2A++ugj4eTkVGk2EhERIYyNjcW9e/eEEPdnAV27dq20nb59+4qJEydW+xxE\nREQIACIzM1O77MMPPxQGBgbaWVp1j+dhRUVFwsDAQBw8ePCR+R/ejrW1tSgpKam0/OEZzAP+/v6i\nS5cu4vXXXxfdunUTarW62v03RpzB6Imo5bn1M2fOwM7ODi+++KJ2WevWrWFnZ4fTp09rlzk6OsLS\n0lJ7287ODtevX3/qnB988AHeffdd9O7dGwsXLkRmZuZjs3p5ecHA4P9+rLy9vVFWVqY9BwQAbm5u\nle5Xm6wmJiZwdnbW3m7ZsiVatWoFa2vrSsse3k52djZGjhwJJycnNGvWDC1btoRGo6nVuS1XV1eY\nmJjUuN4333yDrKwsHDhwANHR0VAqlTXepzFhwehJmzZtoFAocObMmceuJ4So9qTsw8uNjIyqjGk0\nmsdu28DAoErRlZeXV7o9b948nD59Gv7+/jh8+DDc3NywYcMGnWf986tUCoWixu0MHjwYBQUFWLt2\nLY4dO4bU1FQYGhpqD5Eex8zMrMZ1gPuvzt2+fRslJSW4fPlyre7TmLBg9MTKygp+fn5YvXo1ioqK\nqowXFhYCANq3b4/Lly9DpVJpxy5cuIArV66gffv2T5XBxsamyvmYtLS0Kuu1adMG06ZNw759+zBx\n4kR88803j9xe+/btceTIkUq/5IcOHYKxsTGcnJyeKuuTunnzJs6cOYOPPvoIffv2Rbt27XD37l2o\n1WrtOsbGxgDuv9z+VxQWFmLMmDGYMWMGpk6ditGjR+POnTt1kr+hYMHo0ddffw0hBDp37ozt27fj\n7NmzyMzMRFhYmPYwom/fvnB3d8c777yDEydO4Pjx43jnnXfQsWNH9O7d+6n237t3b6SmpmLDhg3I\nysrC559/jl9//VU7/scff2Dq1KlISEiASqXCsWPHcOjQoWqLbcqUKbhy5QqmTJmCM2fOYN++fZgz\nZw6CgoJgamr6VFmfVPPmzWFtbY3w8HBkZWUhMTERkydPrjQTatGiBZo2bYoff/wR+fn5uH379hPt\nY/LkybC2tsb8+fOxdOlSWFhYYOrUqXX9UOo1FowevfTSS0hJSUG/fv0we/ZsuLm5oXfv3ti9ezfW\nrl0L4P60f9euXbCxsUGvXr3g6+uLVq1aYdeuXU99PYufnx9CQ0MREhKCTp06QaVSYcqUKdpxpVKJ\n33//HWPHjoWzszMCAgLg5eWFFStWPHJ79vb2iImJQWpqKjw8PDBhwgSMGDECixYteqqcf4WBgQG2\nbduG9PR0uLq6YurUqfjss88qnVcxNDTEV199hW+++QZ2dnZ48803a739qKgo7N69G5s3b4aRkRFM\nTEzw3//+F99++y22bNki4yHVS7ySl4ik4QyGiKRhwRCRNCwYIpKGBUNE0jwT76YuKSlBRkYGbGxs\neCUkUT3z4GM7XF1dq7xP7ZkomIyMDLzzzjv6jkFET2Hz5s1VPlfnmSgYGxsbAPcDNobPF2mMnJ2f\n7qpjenYplUrY2rbQ/h4/7JkomAeHRa1atYKDg4Oe05AMf/VyfKo/HnV6gyd5iUgaFgwRScOCISJp\nWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYF\nQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBE\nJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TS\nsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBVNHDhw4gCFDhsDe3h4KhQKRkZHasfLy\ncsyePRtubm4wMzODra0tRo4ciYsXL1bZTlJSEvr16wdzc3NYWFigW7duuHHjhg4fCdXGzJkzcOjQ\nAVy/fhV5eSrs2LEd7du3144bGhpiwYLPkJx8DDdvXkdOTjY2bozA88876DG17rFg6khRURFcXV3x\n5ZdfomnTppXGiouLkZKSgpCQEKSkpOD7779HXl4eBgwYALVarV3v2LFj6N+/P3r16oWjR4/ixIkT\nmDFjBoyMjHT9cKgGPXr4YN26cPTq1QcDBgyCWq3GDz/sRfPmzQEApqam8PT0wNKln6Nr1+4IDBwG\nBwd77N69C0qlUs/pdUchhBD6DnHp0iX06dMH8fHxcHCo/w1vbm6O1atXY9y4cdWuc/r0abi4uCA9\nPR0dOnQAAHTr1g2+vr5YuHChjpLqTpMmZvqOIJWZmRmuX7+KwMC38cMPMY9c55VXXkFa2gl06vQa\nfvvtNx0nlEepVMLBwfaRv7+cwejJnTt3AED7P97169dx5MgR2NrawtvbGy1btoSPjw/i4+P1GZNq\nycLCAkqlEoWFhdWu06yZBQCgsPB3XcXSOxaMHpSVleHDDz/E4MGDtY1/4cIFAEBoaCgmTJiA2NhY\n+Pj4wM/PDydPntRnXKqF5cs/R1raSRw9euyR40ZGRli6dDH27t2Hy5ev6Did/hjqO0Bjo1arMWrU\nKBQWFmL37t3a5RqNBgDw3nvvYcKECQAAT09PJCQkYM2aNQgLC9NLXqrZ0qVL0K1bN/Tp01f7fXyY\nUqlERMR6WFpaYujQYXpIqD9SZzDR0dF466234Orqijlz5sjcVb2gVqsxYsQIpKenIz4+Hs8995x2\nzNbWFgAqvRIBAO3atXvkq030bPj886UYNiwQAwa8jpwcVZVxpVKJTZsi0aGDKwYOfAO3bt3SfUg9\nkjqDadGiBaZMmYKDBw+itLRU5q6eeeXl5Rg+fDgyMjKQkJCAVq1aVRp/8cUXYWdnh7Nnz1Zafu7c\nOe1JYHq2LF++DIGBf0f//gNw7ty5KuOGhoaIitoIF5f26N9/IPLz8/WQUr+kFkz//v0BAKdOnWrw\nT25RURGysrIA3D/cuXjxItLS0mBlZQU7OzsEBgYiOTkZe/bsgUKhwLVr1wAAlpaWaNq0KRQKBWbO\nnInQ0FC4ubnB09MT//vf/3D06FGsXr1anw+NHmHlyhUYOXIEhg0bjsLCQrRs2RLA/Z+De/fuQalU\n4r//jUanTp0wdGgghBDadW7fvo2SkhJ9xtcZnoOpI8ePH4evr6/2dmhoKEJDQzF27FjMmzcP33//\nPQCgU6dOle4XERGhfTl7+vTp2hPAN2/ehIuLC2JiYuDu7q6zx0G1M3nyewCA2NgfKi1fsGAhFixY\nBAcHewwZMhgAcPTor5XWmTTpPURFResmqJ6xYOpIr1698LhLimp7udGsWbMwa9asuopFktR0XU9u\n7sUGf+1PbfBlaiKShgVDRNJIPURSq9WoqKiARqNBRUUFSktLoVQqYWjIIzOixkDqDCYsLAxubm5Y\nt24ddu/eDTc3N14wRtSISJ1KBAcHIzg4WOYuiOgZxnMwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCI\nSBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0Lhoik\nYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBrD\n6gbu3Lnz2Ds2a9aszsMQUcNSbcG4uLhAoVBACKFd9uC2QqHAxYsXdRKQiOqvagsmLy9PlzmIqAGq\n1TmYrVu3YtGiRQCAS5cu4cSJE1JDEVHDUGPBBAUFYf/+/YiKigIAmJqaYvLkydKDEVH9V+0h0gOH\nDx9GSkoKPD09AQBWVlYoKyuTHoyI6r8aZzBGRkbQaDRQKBQAgJs3b8LAgK9uE1HNamyKqVOnYujQ\noSgoKEBoaCi8vb0xe/ZsXWQjonquxkOkMWPGoFOnTvj5558BANu3b4erq6v0YERU/9VYMABQUVEB\nIyMjKBQKaDQa2ZmIqIGo8RBp4cKFGDFiBK5cuYJLly5h5MiRWLx4sS6yEVE9V+MMJjo6GidOnICp\nqSkAICQkBJ06dcLcuXOlhyOi+q3GGYyjoyPUarX2tlqtRuvWraWGIqKGodoZzPvvvw+FQgFTU1O4\nuLjAz88PCoUCcXFx8Pb21mVGIqqnqi2YB68Uubi4YNCgQdrlXbt2lZ+KiBqEagtm4sSJusxBRA1Q\njSd5s7OzERISgtOnT6OkpES7/Ny5c1KDEVH9V+NJ3nHjxmH8+PEQQiAmJgbDhg3D8OHDdZGNiOq5\nGgumuLgYfn5+AAAnJycsWLAA+/fvlx6MiOq/Gg+RTExMIISAk5MT1qxZA3t7e1y/fl0X2Yionqux\nYL744gsUFRXhq6++QkhICG7fvo0NGzboIhsR1XM1FkyXLl0AABYWFtoPnSIiqo1qCyYgIED7GTCP\nsnPnTimBiKjhqLZggoKCdJkDAODjMxBA9aVG9VdpabG+I5AkhobVHwhVO9KnTx8pYYio8eBnXxKR\nNCwYIpKm1gVTWloqMwcRNUA1FkxSUhI6dOiANm3aAABOnjyJ4OBg6cGIqP6rsWCmTZuGvXv34rnn\nngMAuLu7860CRFQrNRaMRqOBo6NjpWVKpVJaICJqOGq8kvf5559HUlISFAoFKioqsGrVKrRt21YX\n2YionqtxBhMWFoYVK1bg4sWLaNmyJY4ePYqwsDBdZCOieq7GGUyLFi2wdetWXWQhogamxoKZNGnS\nI9+TtG7dOimBiKjhqLFg+vbtq/26pKQE3333HZ5//nmpoYioYaixYN5+++1Kt0ePHo1+/fpJC0RE\nDccTv1UgJycHubm5MrIQUQNT4wymefPm2nMwGo0GVlZWWLJkifRgRFT/PbZghBA4efIk7O3tAQAG\nBgaP/RAqIqKHPfYQSaFQICAgAEqlEkqlkuVCRE+kxnMwr732GlJSUnSRhYgamGoPkdRqNQwNDXHo\n0CGEh4fDyckJZmZmEEJAoVCwdIioRtUWzIOZy65du3SZh4gakGoLRggB4P5fcyQi+iuqLZiCggKs\nWLGi2jt+8MEHUgIRUcNRbcFUVFSgqKhIO5MhInpS1RaMra0tPvnkE11mIaIGptqXqTlzIaKnVW3B\nxMfH6zIHETVA1RaMlZWVLnMQUQPEP7xGRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgi\nkoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJp\nWDBEJA0LhoikYcEQkTQsGElGj34bMTHfIj39MNLTD2PHjij4+vo8ct1Fiz5BTk46Jk0aq+OU9DR8\nfHzw/fff49KlSxBCYOzYyt8/MzMzfPXVV8jLy0NxcTEyMzMxffp0PaXVj2r/NjU9nWvX8rFkyUqo\nVLkwMDDA0KFDsHbtSgwZMhyZmee16w0c2A9ubi64di1fj2nprzA3N0dGRgY2bdqETZs2VRlfsWIF\n+vbti9GjRyMnJwc9evRAeHg4bty4gejoaD0k1j3OYCT56acEJCYeQm5uHnJycrF8+Srcu1cMT093\n7Tr29rb45JNZ+Ne/5kCtVusxLf0VMTExCAkJwY4dO6DRaKqMd+vWDVFRUUhISEBubi6ioqJw9OhR\ndOnSRQ9p9YMFowMGBgZ4440BMDU1RUrKSQCAUqnEl18uxerV4cjOztFzQpLh0KFDGDx4MBwcHAAA\nXl5e8PDwQGxsrJ6T6Q4PkSRydm6DHTuiYGJijOLiYkyePB1nz94/PHr//SkoLLyNzZv/p+eUJMu0\nadOwZs0a5OXloby8HAAQHByMffv26TmZ7rBgJLpwIQeDBgWiWTMLDBjQF8uXL8CIERPRvLklhg4d\ngkGDhuk7IkkUHByM7t27Y/DgwcjNzUWPHj2wfPlyqFQq/Pjjj/qOpxPSCqasrAzz5s3DkSNHUFhY\nCEdHR7z//vvo2bOnrF0+c8rL1cjNzQMAnDp1Gm5urpgwYRSuXs1HixY2OHYsXruuoaEhZs+ejvHj\nR6Fbt376ikx1pEmTJli8eDECAwOxd+9eAMCpU6fg4eGBGTNmsGCellqthq2tLaKiomBnZ4fExERM\nnz4de/bs0R6TNjYGBgYwMTFGdPQ2xMT8VGls48Yw7NkTi61bd+gpHdUlIyMjGBsbo6KiotLyiooK\nGBg0nlOf0grG1NQUwcHB2tu+vr5wcHDAb7/91igKZtasf2H//oO4cuUazM3NMGTIQHTt2hkTJgTh\n5s1buHnzVqX11Wo1Cgpu4MIFlX4C0xMzMzPDyy+/DOD+fx4vvPAC3N3dcevWLeTl5SEhIQFLlixB\nUVERcnNz0bNnT4wZMwazZs3Sc3Ld0dk5mBs3bkClUmm/IQ2djY01vvhiEaytrXH3bhEyM89h/Pgp\nOHDgsL6jUR3p3LkzEhIStLfnz5+P+fPnIzIyEuPHj8fw4cOxePFibN68GVZWVsjNzcXHH3+M1atX\n6y+0jimEEEL2TsrLyzFp0iS88MILmD9/fpXxS5cuoU+fPigrMwagkB2H9EClOqXvCCSJoaEhWrdu\njfj4+CpHJ9IPBjUaDWbNmgUjIyN8/PHHsndHRM8QqYdIQgiEhITgxo0bCA8Ph5GRkczdEdEzRmrB\nhIaGIjs7GxEREWjSpInMXRHRM0hawVy+fBnbtm2DsbExvL29tcs//fRTDBkyRNZuiegZIq1g7O3t\ncfbsWVmbJ6J6oPFc8UNEOseCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEi\naVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKG\nBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgw\nRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE\n0rBgiEgaFgwRScOCISJpWDBEJA0LhoikMdR3AACoqKj4/18JveYgeQwNn4kfNZLgwff2/36PHxrT\ndZhHKSgoAAAYG5frOQnJ0rp1a31HIMkKCgrg6OhYaZlCCKH3aUNJSQkyMjJgY2MDpVKp7zhE9AQq\nKipQUFAAV1dXNGnSpNLYM1EwRNQw8SQvEUnDgiEiaVgwRCQNC4aIpGHB6Fh0dDTeeustuLq6Ys6c\nOfqOQ3WorKwMH330EXx9feHp6Ql/f38kJibqO5ZePRPXwTQmLVq0wJQpU3Dw4EGUlpbqOw7VIbVa\nDVtbW0RFRcHOzg6JiYmYPn069uzZAwcHB33H0wsWjI71798fAHDq1Cnk5+frOQ3VJVNTUwQHB2tv\n+/r6wsHBAb/99lujLRgeIhFJcuPGDahUKrz88sv6jqI3LBgiCcrLyzFjxgwEBATAyclJ33H0hgVD\nVMc0Gg1mzZoFIyMjfPzxx/qOo1c8B0NUh4QQCAkJwY0bNxAeHg4jIyN9R9IrFoyOqdVqVFRUQKPR\noKKiAqWlpVAqlfw4gwYiNDQU2dnZiIiIqPLGv8aIb3bUsVWrVmH16tWVlgUFBVV69YHqp8uXL6N3\n794wNjau9B/Gp59+iiFDhugxmf6wYIhIGp7kJSJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYBoo\npVIJDw8PuLq6IjAwEMXFxX95WwkJCXjjjTcAALt378aSJUuqXbewsBBff/31E+9j3rx5WL58ea2X\nP2zcuHH49ttva70vlUoFV1fXJ85IT44F00A1bdoUaWlpyMjIgLGxMdasWVNpXAgBjUbzxNsdMmTI\nYz8o668WDDVMLJhGwMfHB1lZWVCpVGjXrh2mTJmCjh07Ii8vD3FxcfDy8kLHjh0RGBiIoqIiAEBs\nbCxeeeUVeHt7Y+fOndptRUZGIigoCACQn5+PgIAAuLu7w93dHYcPH8acOXOQnZ0NDw8PzJw5EwCw\nbNkyvPrqq3Bzc0NoaKh2WwsXLoSzszP69u2Ls2fP1vg4wsPD8eqrr8Ld3R1Dhw6tNCv7+eef4ePj\ng7Zt22Lv3r0A7v+9npkzZ2r3vXbt2qd/MumJsGAaOLVajZiYGHTo0AEAcPbsWYwZMwapqakwMzPD\nggUL8PPPPyMlJQWdO3fGihUrUFJSgkmTJmHPnj04ePAgrl279shtT5s2DT179sTJkyeRkpICFxcX\nLFmyBE5OTkhLS8OyZcsQFxeH8+fPIykpCWlpaThx4gQOHDiAEydOYOvWrUhNTcXOnTuRnJxc42N5\n6623kJycjJMnT6Jdu3ZYv369dkylUiExMRH79u3D5MmTUVJSgvXr18PS0hLJyclITk5GeHg4cnJy\n6uaJpVrhO+waqD/++AMeHh4A7s9gJk6ciCtXrsDR0RFdu3YFABw9ehSnT59G9+7dAdz/TFkvLy9k\nZmbipZdeQps2bQAAo0aNwrp166rs45dffsGmTZsA3D/nY2lpid9//73SOnFxcYiLi4OnpycAoKio\nCOfPn8fdu3cREBAAU1NTAKjVe3UyMjLw73//G4WFhSgqKoKfn592bNiwYTAwMECbNm3QunVrZGZm\nIi4uDunp6drzM7dv38b58+fRtm3b2j+R9FRYMA3Ug3Mwf2ZmZqb9WgiBfv36YcuWLZXWSUtLg0Kh\nqJMcQgjMnTsX7733XqXlK1eufOJ9jBs3Drt27YK7uzsiIyORkJCgHfvzthQKBYQQWLVqVaUiAu7P\ndkg3eIjUiHXt2hW//vorsrKyAADFxcU4d+4cXnnlFeTk5CA7OxsAqhTQA3369EFYWBiA++c77ty5\nAwsLC9y9e1e7jp+fHzZs2KA9t3P58mVcv34dPXr0wHfffYc//vgDd+/exZ49e2rMe/fuXdja2qK8\nvBybN2+uNLZ9+3ZoNBpkZ2fjwoULcHZ2hp+fH8LCwlBeXg4AOHfuHO7du/eEzxI9Dc5gGjEbGxtE\nRkZixIgR2r9wsGDBArRt2xbr1q3DoEGDYG1tDW9vb2RkZFS5/5dffol//OMfWL9+PZRKJcLCwuDl\n5YXu3bvD1dUVAwcOxLJly3DmzBl4eXkBAMzNzREdHY2OHTvi7bffhoeHBxwdHeHj41Nj3s8++wxd\nunSBo6MjOnToUKnInJ2d0bNnT+Tn52PNmjVo0qQJ3n33XahUKnTs2BFCCNjY2GDXrl119OxRbfDj\nGohIGh4iEZE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKT5f2D0lZgDI8S2AAAAAElF\nTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11757e4d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "pipeline = Pipeline([\n",
    "        ('ss', StandardScaler()),\n",
    "        ('pca', PCA(random_state=42)),\n",
    "        ('clf', svc)\n",
    "    ])\n",
    "grid_search = GridSearchCV(pipeline, svc_parameters, verbose=1, cv=5)\n",
    "grid_search.fit(data_train, label_train)\n",
    "print \"Logistic Regression Gridsearch\"\n",
    "print \"Grid Search Best Score:\", grid_search.best_score_\n",
    "print \"Grid Search Best Params:\", grid_search.best_params_\n",
    "print \"Grid Search Best Estimator:\", grid_search.best_estimator_\n",
    "    \n",
    "predictions = grid_search.best_estimator_.predict(data_test)\n",
    "print \"Classification Report:\", classification_report(label_test, predictions)\n",
    "print 'Accuracy: {0:.3f}'.format(accuracy_score(label_test, predictions))  \n",
    "    \n",
    "cnf_matrix = confusion_matrix(label_test, predictions)\n",
    "plot_confusion_matrix(cnf_matrix, classes=grid_search.best_estimator_.classes_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Gridsearching the support vector classifier parameters did not show any improvements in the score of the model, in fact the parameters caused the model to detoriate slightly in accuracy. The grid search resulted in using less principal components as well which may be the cause of the decrease."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using Gradient Boosting in XGBoost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using Sci-Kit learn's prebuilt machine learning packages are generally easy to implement but XGBoost tends to produce similar or better results using ensemble gradient boosting. It also has regularization built in, hence it tends to perform better than standard gradient boosting models."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# SCALE THE DATA - NO PIPELINE OR PCA BEING USED\n",
    "ss = StandardScaler()\n",
    "dataprocessed = ss.fit_transform(dataprocessed)\n",
    "\n",
    "# TRAIN AND TEST SPLIT WITH TEST SIZE OF 20%\n",
    "data_train,data_test,label_train,label_test = train_test_split(dataprocessed,label,test_size = 0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# SET FIXED PARAMETERS - BINARY FOR OUR CLASSIFICATION\n",
    "params_fixed = {\n",
    "    'objective': 'binary:logistic',\n",
    "    'silent': 1,\n",
    "}\n",
    "\n",
    "# HYPERPARAMETER SPACE\n",
    "space = {\n",
    "    'max_depth': (1, 5),\n",
    "    'learning_rate': (10**-4, 10**-1),\n",
    "    'n_estimators': (10, 200),\n",
    "    'min_child_weight': (1, 20),\n",
    "    'subsample': (0, 1),\n",
    "    'colsample_bytree': (0.3, 1)\n",
    "}\n",
    "\n",
    "# DEFINE CLASSIFIER\n",
    "reg = XGBClassifier(**params_fixed)\n",
    "\n",
    "# CREATE A FUNCTION THAT RETURNS A CROSS VALIDATED SCORE FOR PARAMETER TESTING\n",
    "def objective(params):\n",
    "    \"\"\" Wrap a cross validated inverted `accuracy` as objective func \"\"\"\n",
    "    reg.set_params(**{k: p for k, p in zip(space.keys(), params)})\n",
    "    return 1-np.mean(cross_val_score(\n",
    "        reg, data_train, label_train, cv=5, n_jobs=-1,\n",
    "        scoring='accuracy')\n",
    "    )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best accuracy score = 0.73875\n",
      "Best parameters = {'colsample_bytree': 1.0, 'learning_rate': 0.067929683947243275, 'min_child_weight': 1, 'n_estimators': 134, 'subsample': 1, 'max_depth': 5}\n"
     ]
    }
   ],
   "source": [
    "# USES SCI-KIT OPTIMIZE OVER GRIDSEARCH TO FIND THE OPTIMAL HYPERPARAMETERS IN THE HYPERPARAMETER SPACE\n",
    "from skopt import gp_minimize\n",
    "res_gp = gp_minimize(objective, space.values(), n_calls=50, random_state=42)\n",
    "best_hyper_params = {k: v for k, v in zip(space.keys(), res_gp.x)}\n",
    "\n",
    "print \"Best accuracy score =\", 1-res_gp.fun\n",
    "print \"Best parameters =\", best_hyper_params"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# SET THE BEST HYPERPARAMETERS\n",
    "params = best_hyper_params.copy()\n",
    "params.update(params_fixed)\n",
    "\n",
    "# REDEFINE THE CLASSIFIER WITH BEST PARAMETERS\n",
    "clf = XGBClassifier(**params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# FIT THE CLASSIFIER AND PREDICT\n",
    "clf.fit(data_train, label_train)\n",
    "y_test_preds = clf.predict(data_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report:              precision    recall  f1-score   support\n",
      "\n",
      "          1       0.79      0.91      0.85       140\n",
      "          2       0.68      0.45      0.54        60\n",
      "\n",
      "avg / total       0.76      0.77      0.75       200\n",
      "\n",
      "Accuracy: 0.770\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAARgAAAEvCAYAAACe8d6ZAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHoNJREFUeJzt3XlcVOXiBvBn2BkgE0FFKK6ikIqASyqF111SU0Ez15Su\nWl5B86blQoWUmWWXFv2JiQql5tZFUrsaV1TSLEUWlUQUYmLREDMQRJZh3t8fXudKgGDyzsDwfD8f\nPh/mvGfOeWaMp/ecM4tCCCFARCSBkb4DEJHhYsEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgmlBVqxY\ngXbt2kGhUCAqKuqht6dSqaBQKHDmzJmHD9cMDBo0CEFBQfqO0awo+DoY/crPz8eqVatw4MAB5Obm\nws7ODh4eHpg/fz5GjRrVaPtJTU1Fjx49EB0dDW9vb7Rq1QqWlpYPtc2qqioUFBTAzs4OJiYmjZRU\nd1QqFTp27IiEhAT06dOn3vVv3LgBU1NT2NjY6CCdYWh+/1UYEJVKhaeffho2NjZ477334OnpCY1G\ng7i4OMydOxfZ2dmNtq+MjAwAgJ+fHxQKRaNs09jYGO3bt2+UbTVlFRUVMDMzg62trb6jND+C9Gbk\nyJHCwcFBFBcX1xi7ceOG9vdffvlF+Pn5CWtra2FtbS38/f1FTk6OdjwkJER0795d7NixQ3Tq1ElY\nW1uLcePGiYKCAu04gGo/Qggxc+ZMMXr06Gr7vbutu86dOyeGDBkibGxshLW1tfDw8BBHjhwRQgiR\nlZUlAIiEhATt+vHx8aJv377C3NxctG3bVixcuFCUl5drxwcOHCj+/ve/i2XLlok2bdoIe3t7sWjR\nIlFVVVXn8xQZGSmsrKzEv//9b+Hm5iYsLS3FmDFjRGFhodizZ4/o3LmzeOSRR8T06dNFaWmp9n4H\nDx4UPj4+4tFHHxWtW7cWI0aMEBcuXNCO//E5GThwYLXnZfXq1cLR0VHY29trswcGBgohhLh48aJQ\nKpXi888/r7Y/U1NT8cMPP9T5WFoanoPRkxs3buDQoUMICgqCtbV1jfHWrVsDAIQQ8PPzQ35+Po4c\nOYKjR4/iypUr8PPzg7jn6FalUmHXrl3Yu3cvYmNjkZycjODgYADA4sWLERERAQC4evUqrl692uCc\nU6dOhYODA06fPo3k5GSsWLECFhYWta6bl5eHkSNHomfPnkhOTsbmzZuxY8cOLFu2rNp627dvh4mJ\nCU6ePIl169bh448/xq5du+6bo7y8HP/85z+xfft2xMXF4cyZM3juuefw+eef41//+hdiYmJw4MAB\nrF+/XnufW7duYeHChTh9+jSOHTuGVq1aYcyYMaioqAAAnD59GgBw6NAhXL16FdHR0dr7xsfH49y5\nczh06BDi4uJq5HFzc8NHH32EoKAgZGZmoqCgAAEBAQgODkb//v0b9uS2BPpuuJbq1KlTAoCIjo6+\n73qxsbHCyMhIZGVlaZdlZmYKhUIh/vOf/wgh7sw6zM3NRWFhoXadlStXChcXF+3tPXv2iD/+czdk\nBmNjYyOioqJqzfbHGczy5cuFi4tLtdlIZGSkMDMzE7du3RJC3JkF9O/fv9p2hg0bJmbNmlXncxAZ\nGSkAiIsXL2qXLVq0SBgZGWlnaXU9nnuVlJQIIyMjcfz48Vrz37sdOzs7UVZWVm35vTOYu/z8/ES/\nfv3EqFGjxFNPPSXUanWd+2+JOIPRE9HAc+tpaWno0KED/vKXv2iXderUCR06dMCFCxe0y5ydndGq\nVSvt7Q4dOuDatWsPnfPVV1/F7NmzMWTIELz77ru4ePHifbN6e3vDyOh//1n5+PigoqJCew4IADw8\nPKrdryFZzc3N4ebmpr3drl07tG/fHnZ2dtWW3budzMxMTJ06FS4uLnjkkUfQrl07aDSaBp3bcnd3\nh7m5eb3rbdq0CRkZGfjuu++wbds2GBsb13ufloQFoyddunSBQqFAWlrafdcTQtR5Uvbe5aampjXG\nNBrNfbdtZGRUo+gqKyur3V6xYgUuXLgAPz8/nDx5Eh4eHtiyZYvOs/7xKpVCoah3O2PGjEFBQQE+\n++wznDp1CsnJyTAxMdEeIt2PlZVVvesAd67OFRUVoaysDHl5eQ26T0vCgtETW1tb+Pr6Yt26dSgp\nKakxXlhYCADo1q0b8vLyoFKptGM///wzrly5gm7duj1UBnt7+xrnY1JSUmqs16VLFyxYsADffPMN\nZs2ahU2bNtW6vW7duuGHH36o9kd+4sQJmJmZwcXF5aGyPqjffvsNaWlpWL58OYYNG4auXbuiuLgY\narVau46ZmRmAO5fb/4zCwkLMmDEDixcvRmBgIF544QXcvHmzUfIbChaMHq1fvx5CCPTp0wd79uxB\neno6Ll68iPDwcO1hxLBhw+Dp6Ylp06YhMTERZ86cwbRp09CrVy8MGTLkofY/ZMgQJCcnY8uWLcjI\nyMAHH3yA77//Xjt++/ZtBAYG4tixY1CpVDh16hROnDhRZ7HNmzcPV65cwbx585CWloZvvvkGS5cu\nRVBQEJRK5UNlfVCtW7eGnZ0dIiIikJGRgfj4eMydO7faTKht27awtLTEt99+i/z8fBQVFT3QPubO\nnQs7Ozu8/fbbeP/992FjY4PAwMDGfijNGgtGjzp27IikpCQMHz4cS5YsgYeHB4YMGYJ9+/bhs88+\nA3Bn2h8TEwN7e3sMGjQIgwcPRvv27RETE/PQr2fx9fVFSEgIgoOD0bt3b6hUKsybN087bmxsjN9/\n/x0zZ86Em5sb/P394e3tjbCwsFq35+joiIMHDyI5ORleXl7429/+hilTpmDVqlUPlfPPMDIywq5d\nu3Du3Dm4u7sjMDAQ77zzTrXzKiYmJvj000+xadMmdOjQAePGjWvw9rdu3Yp9+/Zh+/btMDU1hbm5\nOb788kt89dVX2LFjh4yH1CzxlbxEJA1nMEQkDQuGiKRhwRCRNCwYIpKmSbybuqysDKmpqbC3t+cr\nIYmambsf2+Hu7l7jfWpNomBSU1Mxbdo0fccgooewffv2Gp+r0yQKxt7eHsCdgC3h80Vaoo4dO+o7\nAkliYmKCxx9/XPt3XG1MD3lquHtY1L59ezg5Oek5Dclw70v0yTDVdnqDJ3mJSBoWDBFJw4IhImlY\nMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVD\nRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQk\nDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKw\nYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYF00i+++47jB07Fo6OjlAoFIiKitKOVVZW\nYsmSJfDw8ICVlRUcHBwwdepUZGdna9dRqVRQKBS1/qxZs0YPj4jqM2DAAHz99dfIzc2FEAIzZ86s\nNv72228jLS0NJSUluHHjBg4fPgxvb289pdUPFkwjKSkpgbu7Oz755BNYWlpWGystLUVSUhKCg4OR\nlJSEr7/+Gjk5OXjmmWegVqsBAI899hiuXr1a7Wf9+vVQKBR47rnn9PGQqB7W1tZITU3FK6+8gtLS\n0hrj6enpCAwMRI8ePeDj44OsrCwcOnQIbdu21UNaPRFNQE5OjnB1dRU5OTn6jtIorKysRGRk5H3X\n+emnnwQAce7cuTrXGTZsmBg+fHgjp9MPAAb9U1xcLGbOnHnfdWxsbIQQQowYMULveRvzx8TEpM6/\nXxOQXty8eRMA0Lp161rHs7KyEBcXh927d+syFkliamqKl156CUVFRUhJSdF3HJ1hwehBRUUFFi1a\nhDFjxsDJyanWdSIiImBnZ4dx48bpOB01ptGjR2Pnzp1QKpW4evUqhg8fjmvXruk7ls7wHIyOqdVq\nTJ8+HYWFhYiMjKxznaioKAQEBMDU1FTHCakxHT16FF5eXnjqqadw6NAh7N69G+3bt9d3LJ2RWjDb\ntm3D+PHj4e7ujqVLl8rcVbOgVqsxZcoUnDt3DnFxcWjTpk2t6+3fvx9Xr17F7NmzdZyQGltpaSky\nMzNx6tQpzJ49G5WVlS3q31XqIVLbtm0xb948HD9+HOXl5TJ31eRVVlZi8uTJSE1NxbFjx+77f7GI\niAgMHDgQrq6uOkxIumBkZARzc3N9x9AZqQUzYsQIAMD58+eRn58vc1d6V1JSgoyMDACARqNBdnY2\nUlJSYGtriw4dOmDixIlISEjA/v37oVAo8OuvvwIAWrVqVe2ydnZ2Nr799lt88cUXenkc1HBWVlbo\n3LkzgDvF8fjjj8PT0xM3btxAYWEhXn/9de1s1N7eHoGBgXBycmpZJ+51cYkyLCxMLFmypM5xQ7hM\nffTo0Vov4c2cOVNkZWXVeYnvj5ez33rrLdG6dWtx+/Zt/TwQSep6/M35Z+DAgbU+1sjISGFpaSmi\no6NFXl6eKCsrE3l5eSImJkb07dtX77kb+4eXqXVg0KBBuPN3VLv7jd0rNDQUoaGhjRWLJIqPj4dC\noahzfPz48TpM0zTxKhIRScOCISJppB4iqdVqVFVVQaPRoKqqCuXl5TA2NoaJCY/MiFoCqTOY8PBw\neHh4YOPGjdi3bx88PDwQHh4uc5dE1IRInUrMnz8f8+fPl7kLImrCeA6GiKRhwRCRNCwYIpKGBUNE\n0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQN\nC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBg\niEgaFgwRSWNS18DNmzfve8dHHnmk0cMQkWGps2C6d+8OhUIBIYR22d3bCoUC2dnZOglIRM1XnQWT\nk5OjyxxEZIAadA5m586dWLVqFQAgNzcXiYmJUkMRkWGot2CCgoJw9OhRbN26FQCgVCoxd+5c6cGI\nqPmr8xDprpMnTyIpKQk9e/YEANja2qKiokJ6MCJq/uqdwZiamkKj0UChUAAAfvvtNxgZ8eo2EdWv\n3qYIDAzEhAkTUFBQgJCQEPj4+GDJkiW6yEZEzVy9h0gzZsxA7969cfjwYQDAnj174O7uLj0YETV/\n9RYMAFRVVcHU1BQKhQIajUZ2JiIyEPUeIr377ruYMmUKrly5gtzcXEydOhXvvfeeLrIRUTNX7wxm\n27ZtSExMhFKpBAAEBwejd+/eWLZsmfRwRNS81TuDcXZ2hlqt1t5Wq9Xo1KmT1FBEZBjqnMH84x//\ngEKhgFKpRPfu3eHr6wuFQoHY2Fj4+PjoMiMRNVN1FszdK0Xdu3fH6NGjtcv79+8vPxURGYQ6C2bW\nrFm6zEFEBqjek7yZmZkIDg7GhQsXUFZWpl1+6dIlqcGIqPmr9yRvQEAAXnzxRQghcPDgQTz//POY\nPHmyLrIRUTNXb8GUlpbC19cXAODi4oKVK1fi6NGj0oMRUfNX7yGSubk5hBBwcXHBhg0b4OjoiGvX\nrukiGxE1c/UWzEcffYSSkhJ8+umnCA4ORlFREbZs2aKLbETUzNVbMP369QMA2NjYaD90ioioIeos\nGH9/f+1nwNQmOjpaSiAiMhx1FkxQUJAucwAAhg71A7+qyTA5OnbRdwSS5D7zkLoLZujQoTKyEFEL\nwukCEUnDgiEiaRpcMOXl5TJzEJEBqrdgTp8+jR49eqBLlzsn6c6ePYv58+dLD0ZEzV+9BbNgwQIc\nOHAAbdq0AQB4enryrQJE1CD1FoxGo4Gzs3O1ZcbGxtICEZHhqPeVvI899hhOnz4NhUKBqqoqrF27\nFq6urrrIRkTNXL0zmPDwcISFhSE7Oxvt2rXDjz/+iPDwcF1kI6Jmrt4ZTNu2bbFz505dZCEiA1Nv\nwcyZM6fW9yRt3LhRSiAiMhz1FsywYcO0v5eVlWHv3r147LHHpIYiIsNQb8FMmjSp2u0XXngBw4cP\nlxaIiAzHA79VICsrC7/88ouMLERkYOqdwbRu3Vp7Dkaj0cDW1harV6+WHoyImr/7FowQAmfPnoWj\noyMAwMjI6L4fQkVEdK/7HiIpFAr4+/vD2NgYxsbGLBcieiD1noPp27cvkpKSdJGFiAxMnYdIarUa\nJiYmOHHiBCIiIuDi4gIrKysIIaBQKFg6RFSvOgvm7swlJiZGl3mIyIDUWTBCCAB3vs2RiOjPqLNg\nCgoKEBYWVucdX331VSmBiMhw1FkwVVVVKCkp0c5kiIgeVJ0F4+DggLfeekuXWYjIwNR5mZozFyJ6\nWHUWTFxcnC5zEJEBqrNgbG1tdZmDiAwQv3iNiKRhwRCRNCwYIpKGBUNE0rBgiEgaFgwRScOCISJp\nWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aIpGHBEJE0LBgikoYF\nQ0TSsGCISBoWDBFJw4IhImnq/OpYejhTpz6HyZP94ejoAAC4fDkL4eFbEB//PQDglVdexjPPDEX7\n9u1QWVmJCxfS8cknG5CcfF6fsamBAgNfxsiRI+Di0hEVFRVISkrB6tX/RHr6Ze06ubmXar1vVNR2\nvPFGqK6i6hULRpL8/Gv48MN1UKlyYGRkBD+/0fi//1uDCRNmID09A1lZvyA0dA1yc/NgYWGBgIAp\n2LTpE4wY8Rx+++2GvuNTPby9++KLL7bj7NnzUCgUWLz4FezYEYUhQ0ahsLAIANCz51PV7uPh4Y7P\nP9+IAwf+rY/IeqEQTeBLqHNzczF06FAANjDko7ZTp/6DsLD12LVrb40xKysrJCUdxaxZC3DixI96\nSCfXrVuF+o4glVKpRFpaImbNmofDh4/Wus4HH6xEv359MHDgMzpOJ5dCASiVCsTFxcHJyanamOH+\nNTchRkZGGDVqOJRKJZKTz9UYNzU1waRJfiguLkFaWu3TamrarK2tYGxsjKKim7WOK5VKjB07Cl9+\nuVvHyfSLh0gSubq6YOfOzTA3N0Np6W0EBb2OS5cyteODBvkgLGwlLC0tUFBwHS++GMTDo2YqNDQY\nqakXkJiYXOu4v/8YmJmZYc+emrNXQ8YZjERZWb/Az286Jk2ahR07/oX33w9Bly6dtOOnTp2Bn990\nTJ48G8eP/4iPP14Fe/s2ekxMf8Zbby3Dk0/2xksvzYdGo6l1nSlTnse33x7GjRu/6zidfkkrmIqK\nCixfvhyDBw9Gz5494efnh/j4eFm7a5IqK9XIzs5FamoawsLWIy3tEgICpmrHb98uQ3Z2Ls6eTUVw\n8Eqo1WpMnDhOj4npQYWELMO4caMxadJMZGfn1LpOt25d4eXVo8UdHgESC0atVsPBwQFbt25FYmIi\nXnnlFSxcuBC5ubmydtnkGRkZwczMtJ5xMx0moocRGhoMP78xmDRpJjIzf65zvWnTnkd2dg6OHz+p\nw3RNg7RzMEqlEvPnz9feHjx4MJycnPDTTz/VONNsiBYtCsSxY9/j11/zYWWlxLPP+qJv3154+eV/\nwMrKCnPmvIAjR46joOA6bG1bY9q059C+fVscPHhY39GpAVauDMGECeMwa9Y8FBUVwd7eDgBw61Yp\nSktLtetZWFjA338swsMj9BVVr3R2kvf69etQqVTo3LmzrnapV3Z2bbBmTSjs7duguLgE6ekZmDNn\nIU6c+BEWFubo3LkTJkwYg0cfbYXCwiKcP38B06a9jPT0DH1HpwYICJgGANi9+4tqy8PC1iIsbK32\n9tixo6BUWmL37mid5msqdPI6mMrKSsyZMwePP/443n777RrjLeV1MC2Zob8OpiXT6+tgNBoNXn/9\ndZiamuLNN9+UvTsiakKkHiIJIRAcHIzr168jIiICpqZ1n+AkIsMjtWBCQkKQmZmJyMhIWFhYyNwV\nETVB0gomLy8Pu3btgpmZGXx8fLTLQ0NDMXbsWFm7JaImRFrBODo6Ij09XdbmiagZ4CUbIpKGBUNE\n0rBgiEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQN\nC4aIpGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBg\niEgaFgwRScOCISJpWDBEJA0LhoikYcEQkTQsGCKShgVDRNKwYIhIGhYMEUnDgiEiaVgwRCQNC4aI\npGHBEJE0LBgikoYFQ0TSsGCISBoWDBFJw4IhImlYMEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEga\nFgwRSWOi7wAAUFVV9d/fNHrNQfIoFPpOQLLc/bf939/x/zSJgikoKPjvb7f0moPkUSrZMIauoKAA\nzs7O1ZYphBBCT3m0ysrKkJqaCnt7exgbG+s7DhE9gKqqKhQUFMDd3R0WFhbVxppEwRCRYeJJXiKS\nhgVDRNKwYIhIGhYMEUnDgtGxbdu2Yfz48XB3d8fSpUv1HYcaUUVFBZYvX47BgwejZ8+e8PPzQ3x8\nvL5j6VWTeB1MS9K2bVvMmzcPx48fR3l5ub7jUCNSq9VwcHDA1q1b0aFDB8THx2PhwoXYv38/nJyc\n9B1PL1gwOjZixAgAwPnz55Gfn6/nNNSYlEol5s+fr709ePBgODk54aeffmqxBcNDJCJJrl+/DpVK\nhc6dO+s7it6wYIgkqKysxOLFi+Hv7w8XFxd9x9EbFgxRI9NoNHj99ddhamqKN998U99x9IrnYIga\nkRACwcHBuH79OiIiImBqaqrvSHrFgtExtVqNqqoqaDQaVFVVoby8HMbGxjAx4T+FIQgJCUFmZiYi\nIyNrvPGvJeKbHXVs7dq1WLduXbVlQUFB1a4+UPOUl5eHIUOGwMzMrNr/MEJDQzF27Fg9JtMfFgwR\nScOTvEQkDQuGiKRhwRCRNCwYIpKGBUNE0rBgiEgaFoyBMjY2hpeXF9zd3TFx4kSUlpb+6W0dO3YM\nzz77LABg3759WL16dZ3rFhYWYv369Q+8jxUrVuDDDz9s8PJ7BQQE4KuvvmrwvlQqFdzd3R84Iz04\nFoyBsrS0REpKClJTU2FmZoYNGzZUGxdCQKN58C+6Gzt27H0/KOvPFgwZJhZMCzBgwABkZGRApVKh\na9eumDdvHnr16oWcnBzExsbC29sbvXr1wsSJE1FSUgIAOHToEJ544gn4+PggOjpau62oqCgEBQUB\nAPLz8+Hv7w9PT094enri5MmTWLp0KTIzM+Hl5YXXXnsNALBmzRo8+eST8PDwQEhIiHZb7777Ltzc\n3DBs2DCkp6fX+zgiIiLw5JNPwtPTExMmTKg2Kzt8+DAGDBgAV1dXHDhwAMCd7+t57bXXtPv+7LPP\nHv7JpAfCgjFwarUaBw8eRI8ePQAA6enpmDFjBpKTk2FlZYWVK1fi8OHDSEpKQp8+fRAWFoaysjLM\nmTMH+/fvx/Hjx/Hrr7/Wuu0FCxZg4MCBOHv2LJKSktC9e3esXr0aLi4uSElJwZo1axAbG4vLly/j\n9OnTSElJQWJiIr777jskJiZi586dSE5ORnR0NBISEup9LOPHj0dCQgLOnj2Lrl27YvPmzdoxlUqF\n+Ph4fPPNN5g7dy7KysqwefNmtGrVCgkJCUhISEBERASysrIa54mlBuE77AzU7du34eXlBeDODGbW\nrFm4cuUKnJ2d0b9/fwDAjz/+iAsXLuDpp58GcOczZb29vXHx4kV07NgRXbp0AQBMnz4dGzdurLGP\nI0eO4IsvvgBw55xPq1at8Pvvv1dbJzY2FrGxsejZsycAoKSkBJcvX0ZxcTH8/f2hVCoBoEHv1UlN\nTcUbb7yBwsJClJSUwNfXVzv2/PPPw8jICF26dEGnTp1w8eJFxMbG4ty5c9rzM0VFRbh8+TJcXV0b\n/kTSQ2HBGKi752D+yMrKSvu7EALDhw/Hjh07qq2TkpICRSN9W70QAsuWLcPLL79cbfnHH3/8wPsI\nCAhATEwMPD09ERUVhWPHjmnH/rgthUIBIQTWrl1brYiAO7Md0g0eIrVg/fv3x/fff4+MjAwAQGlp\nKS5duoQnnngCWVlZyMzMBIAaBXTX0KFDER4eDuDO+Y6bN2/CxsYGxcXF2nV8fX2xZcsW7bmdvLw8\nXLt2DX/961+xd+9e3L59G8XFxdi/f3+9eYuLi+Hg4IDKykps37692tiePXug0WiQmZmJn3/+GW5u\nbvD19UV4eDgqKysBAJcuXcKtW7ce8Fmih8EZTAtmb2+PqKgoTJkyRfsNBytXroSrqys2btyI0aNH\nw87ODj4+PkhNTa1x/08++QQvvfQSNm/eDGNjY4SHh8Pb2xtPP/003N3dMXLkSKxZswZpaWnw9vYG\nAFhbW2Pbtm3o1asXJk2aBC8vLzg7O2PAgAH15n3nnXfQr18/ODs7o0ePHtWKzM3NDQMHDkR+fj42\nbNgACwsLzJ49GyqVCr169YIQAvb29oiJiWmkZ48agh/XQETS8BCJiKRhwRCRNCwYIpKGBUNE0rBg\niEgaFgwRScOCISJpWDBEJM3/A+9E54eTqKjrAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x1153073d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "print \"Classification Report:\", classification_report(label_test, y_test_preds)\n",
    "print 'Accuracy: {0:.3f}'.format(accuracy_score(label_test, y_test_preds))  \n",
    "cnf_matrix = confusion_matrix(label_test, y_test_preds)\n",
    "plot_confusion_matrix(cnf_matrix, classes=grid_search.best_estimator_.classes_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using XGBoost we are able to outperform our grid-searched classifiers in sci-kit learn as expected. A hyperparameter optimized XGBoost Classifier produced an accuracy score of 0.77 on the test data, the highest of all tested classifiers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "** Model Results on Test Data**\n",
    "- Logistic Regression: **0.73**\n",
    "- Random Forest: **0.68**\n",
    "- Support Vector Machines: **0.72**\n",
    "- XGBoost: **0.77**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Potential to improve**\n",
    "- Testing out a Neural Network with Keras or TensorFlow\n",
    "- More feature engineering with our given features, i.e. creating a column with a rank for Purpose, or creating new features from combining existing features such as Purpose and Duration in creative ways.\n",
    "- Test on other similar data to see how well the model holds up as this data is limited to the the German market.\n",
    "- Use Patsy and peel away the PCA/Scaling steps from a pipeline and pull coefficients from the gridsearch to find out how the coefficient weighting differs for each feature."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Using gradient boosted trees in XGBoost's classifier we were able to go as high as **0.77** on classification accuracy on the test data, decreasing false positives and negatives relative to the other models. As the size of the data is quite small, these classification algorithms tend to perform similarly however it would be interesting to see how well a neural network compares in results but it tends to be more computationally intensive and given the size of the data, tuning parameters and more feature engineering is likely to produce similar results."
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python [conda root]",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
